Jul 28 16:00:11.709126 ip-10-0-138-10 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 28 16:00:11.709137 ip-10-0-138-10 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 28 16:00:11.709144 ip-10-0-138-10 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 28 16:00:11.709399 ip-10-0-138-10 systemd[1]: Failed to start Kubernetes Kubelet. Jul 28 16:00:21.489794 ip-10-0-138-10 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 28 16:00:21.489812 ip-10-0-138-10 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot d4108e44460d43a5b71b82c52decaa01 -- Jul 28 16:02:10.827821 ip-10-0-138-10 systemd[1]: Starting Kubernetes Kubelet... Jul 28 16:02:11.199518 ip-10-0-138-10 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 16:02:11.199518 ip-10-0-138-10 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 16:02:11.199518 ip-10-0-138-10 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 16:02:11.199518 ip-10-0-138-10 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 16:02:11.199518 ip-10-0-138-10 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 16:02:11.200219 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.199590 2550 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 16:02:11.202854 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202837 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:02:11.202854 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202851 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:02:11.202854 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202855 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202859 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202862 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202865 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202867 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202877 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202881 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202884 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202887 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202889 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202893 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202898 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202900 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202903 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202906 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202908 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202911 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202914 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202916 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:02:11.202983 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202920 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202922 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202925 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202928 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202930 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202933 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202935 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202938 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202941 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202943 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202946 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202949 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202951 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202954 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202957 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202960 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202962 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202966 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202968 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202971 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:02:11.203648 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202974 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202976 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202979 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202982 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202984 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202987 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202989 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202992 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.202995 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203000 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203003 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203006 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203009 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203012 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203015 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203018 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203021 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203023 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203026 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203029 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:02:11.204353 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203032 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203035 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203038 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203040 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203044 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203046 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203049 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203053 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203056 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203059 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203063 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203066 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203069 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203072 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203075 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203077 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203080 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203082 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203085 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203087 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:02:11.205204 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203090 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203093 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203095 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203097 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203100 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203102 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203105 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203107 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203110 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203112 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203115 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203118 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203120 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203122 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203125 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203127 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203130 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203133 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203848 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:02:11.205914 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203855 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203859 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203862 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203865 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203869 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203871 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203874 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203877 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203880 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203882 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203885 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203888 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203890 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203893 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203896 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203898 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203901 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203904 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203906 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203909 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:02:11.206594 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203911 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203914 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203916 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203919 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203921 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203924 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203926 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203929 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203931 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203934 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203937 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203940 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203943 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203945 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203949 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203953 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203956 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203959 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203962 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:02:11.207317 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203964 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203967 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203969 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203972 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203974 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203977 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203979 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203982 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203984 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203987 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203989 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203992 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203994 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.203997 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204000 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204002 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204004 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204007 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204011 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204013 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:02:11.208013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204016 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204018 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204021 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204025 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204029 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204032 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204035 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204038 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204041 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204044 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204046 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204050 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204055 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204058 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204060 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204064 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204067 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204069 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204072 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:02:11.208802 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204075 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204077 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204080 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204083 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204085 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204087 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204090 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204093 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204095 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204098 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204101 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204104 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204107 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204109 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204112 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204114 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204117 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204120 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204122 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:02:11.209478 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204125 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204457 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204463 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204467 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204470 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204473 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204475 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204478 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204481 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204485 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204488 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204491 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204495 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204497 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204500 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204503 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204505 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204508 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204511 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:02:11.209994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204513 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204516 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204519 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204522 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204526 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204529 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204532 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204534 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204555 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204560 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204571 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204573 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204576 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204578 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204581 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204583 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204586 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204589 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204598 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204602 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:02:11.210490 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204605 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204608 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204610 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204613 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204615 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204618 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204620 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204623 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204625 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204628 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204630 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204633 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204635 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204641 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204643 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204646 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204648 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204652 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204655 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204657 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:02:11.211007 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204660 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204662 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204665 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204668 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204670 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204673 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204675 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204678 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204680 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204683 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204685 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204687 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204690 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204692 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204695 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204697 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204700 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204703 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204706 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:02:11.211505 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204708 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204711 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204714 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204717 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204719 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204722 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204727 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204730 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204732 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204735 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204738 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204741 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204743 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204745 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204748 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204750 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204753 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204755 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204758 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204761 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:02:11.212023 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204763 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.204766 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204827 2550 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204840 2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204846 2550 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204850 2550 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204854 2550 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204857 2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204861 2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204865 2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204868 2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204871 2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204874 2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204877 2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204880 2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204883 2550 flags.go:64] FLAG: --cgroup-root="" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204886 2550 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204889 2550 flags.go:64] FLAG: --client-ca-file="" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204893 2550 flags.go:64] FLAG: --cloud-provider="external" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204895 2550 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204899 2550 flags.go:64] FLAG: --cluster-domain="" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204902 2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204906 2550 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204909 2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 16:02:11.212509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204912 2550 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204916 2550 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204919 2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204922 2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204925 2550 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204928 2550 flags.go:64] FLAG: --contention-profiling="false" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204931 2550 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204934 2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204936 2550 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204940 2550 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204944 2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204947 2550 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204950 2550 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204952 2550 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204955 2550 flags.go:64] FLAG: --enable-server="true" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204959 2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204963 2550 flags.go:64] FLAG: --event-burst="100" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204966 2550 flags.go:64] FLAG: --event-qps="50" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204969 2550 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204974 2550 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204979 2550 flags.go:64] FLAG: --eviction-hard="" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.204984 2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205009 2550 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205022 2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205026 2550 flags.go:64] FLAG: --eviction-soft="" Jul 28 16:02:11.213106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205030 2550 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205034 2550 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205037 2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205040 2550 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205043 2550 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205046 2550 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205049 2550 flags.go:64] FLAG: --feature-gates="" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205054 2550 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205057 2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205061 2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205064 2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205067 2550 flags.go:64] FLAG: --healthz-port="10248" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205071 2550 flags.go:64] FLAG: --help="false" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205074 2550 flags.go:64] FLAG: --hostname-override="ip-10-0-138-10.ec2.internal" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205077 2550 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205080 2550 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205083 2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205087 2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205090 2550 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205094 2550 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205097 2550 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205099 2550 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205103 2550 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 16:02:11.213743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205106 2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205109 2550 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205112 2550 flags.go:64] FLAG: --kube-reserved="" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205115 2550 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205117 2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205121 2550 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205123 2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205128 2550 flags.go:64] FLAG: --lock-file="" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205131 2550 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205134 2550 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205137 2550 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205143 2550 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205146 2550 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205149 2550 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205152 2550 flags.go:64] FLAG: --logging-format="text" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205155 2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205158 2550 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205161 2550 flags.go:64] FLAG: --manifest-url="" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205164 2550 flags.go:64] FLAG: --manifest-url-header="" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205168 2550 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205172 2550 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205176 2550 flags.go:64] FLAG: --max-pods="110" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205179 2550 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205182 2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205185 2550 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 16:02:11.214312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205188 2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205191 2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205195 2550 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205198 2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205206 2550 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205209 2550 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205212 2550 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205216 2550 flags.go:64] FLAG: --pod-cidr="" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205219 2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205224 2550 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205227 2550 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205230 2550 flags.go:64] FLAG: --pods-per-core="0" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205233 2550 flags.go:64] FLAG: --port="10250" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205236 2550 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205240 2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0745c443ab4ad4409" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205247 2550 flags.go:64] FLAG: --qos-reserved="" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205250 2550 flags.go:64] FLAG: --read-only-port="10255" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205253 2550 flags.go:64] FLAG: --register-node="true" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205256 2550 flags.go:64] FLAG: --register-with-taints="" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205259 2550 flags.go:64] FLAG: --registry-burst="10" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205262 2550 flags.go:64] FLAG: --registry-qps="5" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205265 2550 flags.go:64] FLAG: --reserved-cpus="" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205268 2550 flags.go:64] FLAG: --reserved-memory="" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205271 2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205274 2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 16:02:11.214940 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205277 2550 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205280 2550 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205283 2550 flags.go:64] FLAG: --runonce="false" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205285 2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205288 2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205291 2550 flags.go:64] FLAG: --seccomp-default="false" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205294 2550 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205297 2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205300 2550 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205302 2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205305 2550 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205308 2550 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205311 2550 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205314 2550 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205316 2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205319 2550 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205322 2550 flags.go:64] FLAG: --system-cgroups="" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205325 2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205331 2550 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205334 2550 flags.go:64] FLAG: --tls-cert-file="" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205336 2550 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205342 2550 flags.go:64] FLAG: --tls-min-version="" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205346 2550 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205349 2550 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205352 2550 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 16:02:11.215647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205355 2550 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205358 2550 flags.go:64] FLAG: --v="2" Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205361 2550 flags.go:64] FLAG: --version="false" Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205366 2550 flags.go:64] FLAG: --vmodule="" Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205369 2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.205372 2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205488 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205492 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205496 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205499 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205502 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205505 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205507 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205510 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205512 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205515 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205517 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205520 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205522 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205525 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205527 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:02:11.216247 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205530 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205532 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205535 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205548 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205551 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205554 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205556 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205559 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205564 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205566 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205569 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205572 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205575 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205577 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205580 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205582 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205585 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205588 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205591 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205594 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:02:11.216805 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205596 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205601 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205604 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205607 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205610 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205613 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205615 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205618 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205621 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205624 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205626 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205629 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205632 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205635 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205637 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205640 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205643 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205645 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205648 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205651 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:02:11.217281 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205655 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205659 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205662 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205666 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205669 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205672 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205674 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205677 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205679 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205682 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205684 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205687 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205689 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205692 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205695 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205697 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205700 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205702 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205704 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:02:11.217786 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205707 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205710 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205712 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205715 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205717 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205720 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205722 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205725 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205727 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205729 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205732 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205734 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205738 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205742 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205744 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205747 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205749 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205757 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205760 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:02:11.218244 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205763 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:02:11.218743 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205765 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:02:11.218743 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205768 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:02:11.218743 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205770 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:02:11.218743 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205773 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:02:11.218743 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.205775 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:02:11.218743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.206396 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 16:02:11.218743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.212652 2550 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 16:02:11.218743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.212666 2550 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 16:02:11.218743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.212685 2550 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 16:02:11.218743 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.212691 2550 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 16:02:11.218743 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212739 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:02:11.218743 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212743 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:02:11.218743 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212747 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:02:11.218743 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212750 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212752 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212755 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212759 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212761 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212764 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212766 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212769 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212772 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212774 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212777 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212780 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212783 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212787 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212789 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212792 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212795 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212798 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212800 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212803 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:02:11.219097 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212805 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212808 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212810 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212813 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212816 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212818 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212823 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212826 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212830 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212832 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212835 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212837 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212840 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212842 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212845 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212848 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212850 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212853 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212855 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:02:11.219699 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212858 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212861 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212865 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212867 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212870 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212872 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212875 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212878 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212881 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212883 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212886 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212888 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212891 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212893 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212895 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212898 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212901 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212903 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212906 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212908 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:02:11.220283 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212911 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212914 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212916 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212919 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212921 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212924 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212926 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212928 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212931 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212934 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212936 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212939 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212941 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212944 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212946 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212949 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212951 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212954 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212957 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:02:11.221139 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212959 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212963 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212967 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212970 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212973 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212976 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212979 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212982 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212985 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212987 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212990 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212993 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212996 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.212998 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213001 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213003 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213006 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:02:11.221636 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213009 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.213013 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213111 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213117 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213120 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213123 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213127 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213130 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213132 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213135 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213138 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213140 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213143 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213145 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213148 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:02:11.222058 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213150 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213153 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213157 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213161 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213163 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213166 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213169 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213171 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213174 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213177 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213180 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213182 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213185 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213187 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213190 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213193 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213195 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213198 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213200 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213203 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:02:11.222437 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213205 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213210 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213213 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213216 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213219 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213221 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213224 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213226 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213229 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213231 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213234 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213237 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213239 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213241 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213244 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213246 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213249 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213251 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213254 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:02:11.222942 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213256 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213259 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213261 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213264 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213266 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213269 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213271 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213274 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213276 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213280 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213283 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213286 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213289 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213291 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213294 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213296 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213299 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213301 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213304 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213306 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:02:11.223643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213309 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213311 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213314 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213316 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213319 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213321 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213324 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213326 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213329 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213331 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213334 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213336 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213339 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213341 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213344 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213346 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213349 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213351 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213354 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213356 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:02:11.224152 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213359 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:02:11.224679 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213361 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:02:11.224679 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213364 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:02:11.224679 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213366 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:02:11.224679 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213369 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:02:11.224679 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213371 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:02:11.224679 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:11.213376 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:02:11.224679 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.213381 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 16:02:11.224679 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.213467 2550 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 16:02:11.224679 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.215280 2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 16:02:11.224679 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.216610 2550 server.go:1019] "Starting client certificate rotation" Jul 28 16:02:11.224679 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.216702 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 16:02:11.224679 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.216735 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 16:02:11.224679 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.218794 2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 16:02:11.226051 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.226030 2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 16:02:11.239430 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.239410 2550 log.go:25] "Validated CRI v1 runtime API" Jul 28 16:02:11.241125 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.241111 2550 log.go:25] "Validated CRI v1 image API" Jul 28 16:02:11.241903 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.241890 2550 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 16:02:11.245456 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.245436 2550 fs.go:135] Filesystem UUIDs: map[4f616761-5f68-4f1c-a751-3847319ca237:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c9b7d9cc-811a-4252-a1a8-9d5b4dc78c41:/dev/nvme0n1p4] Jul 28 16:02:11.245515 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.245454 2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 16:02:11.250650 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.250551 2550 manager.go:216] Machine: {Timestamp:2026-07-28 16:02:11.249429805 +0000 UTC m=+0.313464596 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3211048 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e4280af3d1ea3d1c963b9d358fc13 SystemUUID:ec2e4280-af3d-1ea3-d1c9-63b9d358fc13 BootID:d4108e44-460d-43a5-b71b-82c52decaa01 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ab:0f:24:a5:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ab:0f:24:a5:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:73:e6:46:9b:ae Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 16:02:11.250650 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.250643 2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 16:02:11.250754 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.250728 2550 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 16:02:11.251491 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.251477 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 16:02:11.251557 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.251522 2550 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 16:02:11.251696 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.251563 2550 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-10.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 16:02:11.252344 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.252334 2550 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 16:02:11.252379 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.252346 2550 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 16:02:11.252379 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.252353 2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 16:02:11.253010 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.252999 2550 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 16:02:11.254039 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.254030 2550 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 16:02:11.256906 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.256895 2550 state_mem.go:36] "Initialized new in-memory state store" Jul 28 16:02:11.257012 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.257002 2550 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 16:02:11.257054 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.257029 2550 kubelet.go:486] "Attempting to sync node with API server" Jul 28 16:02:11.257054 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.257038 2550 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 16:02:11.257054 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.257048 2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 16:02:11.257171 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.257056 2550 kubelet.go:398] "Adding apiserver pod source" Jul 28 16:02:11.257171 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.257063 2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 16:02:11.257352 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.257341 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 16:02:11.257383 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.257358 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 16:02:11.259290 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.259269 2550 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 16:02:11.260627 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.260603 2550 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 16:02:11.260699 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.260643 2550 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 16:02:11.261604 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.261583 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 16:02:11.261688 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.261613 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 16:02:11.261688 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.261623 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 16:02:11.261688 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.261632 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 16:02:11.261688 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.261648 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 16:02:11.261688 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.261664 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 16:02:11.261688 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.261677 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 16:02:11.261688 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.261684 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 16:02:11.261977 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.261695 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 16:02:11.261977 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.261706 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 16:02:11.261977 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.261719 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 16:02:11.261977 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.261727 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 16:02:11.261977 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.261759 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 16:02:11.261977 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.261767 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 16:02:11.264915 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.264898 2550 server.go:1268] "Started kubelet" Jul 28 16:02:11.265125 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.264972 2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 16:02:11.265125 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.265011 2550 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 16:02:11.265125 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.265105 2550 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 16:02:11.265654 ip-10-0-138-10 systemd[1]: Started Kubernetes Kubelet. Jul 28 16:02:11.266718 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.266485 2550 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 16:02:11.267060 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.267045 2550 server.go:310] "Adding debug handlers to kubelet server" Jul 28 16:02:11.270230 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.270187 2550 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 16:02:11.271641 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.271624 2550 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 16:02:11.271873 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.271853 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 16:02:11.272081 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.271227 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-10.ec2.internal.18c67f8999b2dd8a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-10.ec2.internal,UID:ip-10-0-138-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-10.ec2.internal,},FirstTimestamp:2026-07-28 16:02:11.264863626 +0000 UTC m=+0.328898417,LastTimestamp:2026-07-28 16:02:11.264863626 +0000 UTC m=+0.328898417,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-10.ec2.internal,}" Jul 28 16:02:11.272172 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.272110 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 16:02:11.273368 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.273350 2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 16:02:11.273368 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.273362 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 16:02:11.273507 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.273413 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:11.273865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.273838 2550 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 16:02:11.273865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.273850 2550 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 16:02:11.273865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.273857 2550 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 16:02:11.274054 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.273953 2550 reconstruct.go:97] "Volume reconstruction finished" Jul 28 16:02:11.274054 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.273961 2550 reconciler.go:29] "Reconciler: start to sync state" Jul 28 16:02:11.274405 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.274238 2550 factory.go:55] Registering systemd factory Jul 28 16:02:11.274405 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.274280 2550 factory.go:223] Registration of the systemd container factory successfully Jul 28 16:02:11.275475 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.275458 2550 factory.go:153] Registering CRI-O factory Jul 28 16:02:11.275475 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.275475 2550 factory.go:223] Registration of the crio container factory successfully Jul 28 16:02:11.275653 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.275523 2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 16:02:11.275653 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.275558 2550 factory.go:103] Registering Raw factory Jul 28 16:02:11.275653 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.275571 2550 manager.go:1196] Started watching for new ooms in manager Jul 28 16:02:11.276039 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.276023 2550 manager.go:349] Starting recovery of all containers Jul 28 16:02:11.277972 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.277748 2550 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 16:02:11.277972 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.277854 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 16:02:11.280324 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.280299 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v8f26" Jul 28 16:02:11.288222 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.287910 2550 manager.go:354] Recovery completed Jul 28 16:02:11.289385 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.289368 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v8f26" Jul 28 16:02:11.292894 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.292880 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:02:11.294596 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.294581 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:02:11.294639 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.294613 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:02:11.294639 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.294624 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:02:11.295088 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.295074 2550 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 16:02:11.295139 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.295088 2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 16:02:11.295139 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.295095 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:02:11.295139 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.295106 2550 state_mem.go:36] "Initialized new in-memory state store" Jul 28 16:02:11.295748 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.295734 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:02:11.295797 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.295757 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:02:11.295797 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.295766 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:02:11.296007 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.295996 2550 policy_none.go:49] "None policy: Start" Jul 28 16:02:11.296040 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.296011 2550 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 16:02:11.296040 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.296022 2550 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 16:02:11.296943 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.296929 2550 policy_none.go:47] "Start" Jul 28 16:02:11.297015 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.296943 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-10.ec2.internal.18c67f899b7892a4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-10.ec2.internal,UID:ip-10-0-138-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-10.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-10.ec2.internal,},FirstTimestamp:2026-07-28 16:02:11.294597796 +0000 UTC m=+0.358632592,LastTimestamp:2026-07-28 16:02:11.294597796 +0000 UTC m=+0.358632592,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-10.ec2.internal,}" Jul 28 16:02:11.326068 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.326055 2550 manager.go:337] "Starting Device Plugin manager" Jul 28 16:02:11.326139 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.326095 2550 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 16:02:11.326139 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.326104 2550 server.go:87] "Starting device plugin registration server" Jul 28 16:02:11.326317 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.326305 2550 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 16:02:11.326357 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.326321 2550 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 16:02:11.326562 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.326411 2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 16:02:11.326660 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.326628 2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 16:02:11.326660 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.326645 2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 16:02:11.326950 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.326933 2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 16:02:11.327006 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.326971 2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:11.404350 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.404319 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 16:02:11.406339 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.405368 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 16:02:11.406339 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.405399 2550 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 16:02:11.406339 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.405421 2550 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 16:02:11.406339 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.405450 2550 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 16:02:11.408043 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.408027 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:02:11.427067 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.427045 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:02:11.427752 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.427732 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:02:11.427845 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.427760 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:02:11.427845 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.427774 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:02:11.427845 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.427798 2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-10.ec2.internal" Jul 28 16:02:11.437151 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.437131 2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-10.ec2.internal" Jul 28 16:02:11.437229 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.437152 2550 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-10.ec2.internal\": node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:11.469796 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.469747 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:11.506281 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.506260 2550 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-10.ec2.internal"] Jul 28 16:02:11.506363 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.506310 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:02:11.507084 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.507068 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:02:11.507146 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.507093 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:02:11.507146 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.507110 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:02:11.509358 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.509345 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:02:11.509499 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.509478 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal" Jul 28 16:02:11.509533 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.509514 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:02:11.510043 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.510025 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:02:11.510043 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.510039 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:02:11.510147 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.510056 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:02:11.510147 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.510063 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:02:11.510147 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.510071 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:02:11.510147 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.510077 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:02:11.512253 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.512232 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-10.ec2.internal" Jul 28 16:02:11.512367 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.512259 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:02:11.512944 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.512926 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:02:11.513039 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.512956 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:02:11.513039 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.512971 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:02:11.544066 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.544043 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-10.ec2.internal\" not found" node="ip-10-0-138-10.ec2.internal" Jul 28 16:02:11.548176 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.548162 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-10.ec2.internal\" not found" node="ip-10-0-138-10.ec2.internal" Jul 28 16:02:11.570391 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:11.570373 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:11.575631 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.575610 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0d72e1a34529c7526e7d432be101f4be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal\" (UID: \"0d72e1a34529c7526e7d432be101f4be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal" Jul 28 16:02:11.575726 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.575637 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d72e1a34529c7526e7d432be101f4be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal\" (UID: \"0d72e1a34529c7526e7d432be101f4be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal" Jul 28 16:02:11.575726 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:11.575652 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ce1d00f21bc867d7fa218ae3a0c694e1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-10.ec2.internal\" (UID: \"ce1d00f21bc867d7fa218ae3a0c694e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-10.ec2.internal" Jul 28 16:02:12.700727 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:12.700704 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:12.706027 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:12.706011 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0d72e1a34529c7526e7d432be101f4be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal\" (UID: \"0d72e1a34529c7526e7d432be101f4be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal" Jul 28 16:02:12.706108 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:12.706089 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d72e1a34529c7526e7d432be101f4be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal\" (UID: \"0d72e1a34529c7526e7d432be101f4be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal" Jul 28 16:02:12.706164 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:12.706036 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d72e1a34529c7526e7d432be101f4be-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal\" (UID: \"0d72e1a34529c7526e7d432be101f4be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal" Jul 28 16:02:12.706164 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:12.706150 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ce1d00f21bc867d7fa218ae3a0c694e1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-10.ec2.internal\" (UID: \"ce1d00f21bc867d7fa218ae3a0c694e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-10.ec2.internal" Jul 28 16:02:12.706164 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:12.706151 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0d72e1a34529c7526e7d432be101f4be-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal\" (UID: \"0d72e1a34529c7526e7d432be101f4be\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal" Jul 28 16:02:12.706267 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:12.706177 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ce1d00f21bc867d7fa218ae3a0c694e1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-10.ec2.internal\" (UID: \"ce1d00f21bc867d7fa218ae3a0c694e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-10.ec2.internal" Jul 28 16:02:12.801485 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:12.801431 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:12.875978 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:12.875951 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal" Jul 28 16:02:12.881212 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:12.880969 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-10.ec2.internal" Jul 28 16:02:12.901723 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:12.901701 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:13.002476 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:13.002442 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:13.102988 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:13.102933 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:13.203473 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:13.203443 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:13.245864 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:13.245846 2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 16:02:13.246211 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:13.245960 2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 16:02:13.303945 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:13.303925 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:13.323147 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:13.323118 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 15:57:12 +0000 UTC" deadline="2028-01-17 10:02:58.334922089 +0000 UTC" Jul 28 16:02:13.323147 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:13.323144 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12906h0m45.01178118s" Jul 28 16:02:13.404363 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:13.404342 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:13.502277 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:13.502248 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce1d00f21bc867d7fa218ae3a0c694e1.slice/crio-20e9c7e7a7ae3b00e74d5f460bbac807fc9917bc6342fc8d8d1757ed929dba9c WatchSource:0}: Error finding container 20e9c7e7a7ae3b00e74d5f460bbac807fc9917bc6342fc8d8d1757ed929dba9c: Status 404 returned error can't find the container with id 20e9c7e7a7ae3b00e74d5f460bbac807fc9917bc6342fc8d8d1757ed929dba9c Jul 28 16:02:13.502548 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:13.502532 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d72e1a34529c7526e7d432be101f4be.slice/crio-b097564cc36a36c51e01d5db9c84ff7f1533fb5aee2a9fbf9426b7768685cb76 WatchSource:0}: Error finding container b097564cc36a36c51e01d5db9c84ff7f1533fb5aee2a9fbf9426b7768685cb76: Status 404 returned error can't find the container with id b097564cc36a36c51e01d5db9c84ff7f1533fb5aee2a9fbf9426b7768685cb76 Jul 28 16:02:13.504686 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:13.504669 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:13.507878 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:13.507860 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 16:02:13.604889 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:13.604857 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:13.705355 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:13.705295 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-10.ec2.internal\" not found" Jul 28 16:02:13.708270 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:13.708252 2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:02:13.803564 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:13.803535 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal" Jul 28 16:02:13.815416 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:13.815393 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 16:02:13.816939 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:13.816923 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-10.ec2.internal" Jul 28 16:02:13.826495 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:13.826477 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 16:02:13.861087 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:13.861065 2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:02:13.863973 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:13.863943 2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:02:14.287693 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.287660 2550 apiserver.go:52] "Watching apiserver" Jul 28 16:02:14.293503 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.293477 2550 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 16:02:14.293852 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.293830 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-nww4s","kube-system/kube-apiserver-proxy-ip-10-0-138-10.ec2.internal","openshift-cluster-node-tuning-operator/tuned-8xx66","openshift-multus/multus-88478","openshift-multus/network-metrics-daemon-ngzg7","openshift-network-operator/iptables-alerter-zx2dx","openshift-ovn-kubernetes/ovnkube-node-4xnwq","kube-system/konnectivity-agent-9nkv2","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg","openshift-dns/node-resolver-jn6q8","openshift-image-registry/node-ca-pdxnl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal","openshift-multus/multus-additional-cni-plugins-2ldwp","openshift-network-diagnostics/network-check-target-hxxbk"] Jul 28 16:02:14.296375 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.296352 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.298936 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.298523 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.298936 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.298727 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 16:02:14.298936 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.298774 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 16:02:14.298936 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.298860 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 16:02:14.299215 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.299146 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 16:02:14.299215 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.299175 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jwqgk\"" Jul 28 16:02:14.299215 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.299187 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 16:02:14.299468 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.299450 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 16:02:14.300247 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.300231 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 16:02:14.300541 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.300524 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 16:02:14.300753 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.300736 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 16:02:14.300846 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.300829 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qqwmz\"" Jul 28 16:02:14.300906 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.300850 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 16:02:14.301220 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.300956 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 16:02:14.301220 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.301029 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.302958 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.302939 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 16:02:14.303048 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.302945 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 16:02:14.303241 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.303182 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fx55l\"" Jul 28 16:02:14.303241 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.303193 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-88478" Jul 28 16:02:14.304075 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.304056 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 16:02:14.304968 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.304951 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 16:02:14.305189 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.305170 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4pths\"" Jul 28 16:02:14.305894 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.305850 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:14.306007 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:14.305922 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:14.308910 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.308884 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zx2dx" Jul 28 16:02:14.311091 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.311062 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 16:02:14.311189 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.311113 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6882q\"" Jul 28 16:02:14.311925 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.311310 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.311925 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.311364 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 16:02:14.311925 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.311378 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 16:02:14.313918 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.313391 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 16:02:14.313918 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.313436 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 16:02:14.313918 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.313437 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 16:02:14.313918 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.313567 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5j8jv\"" Jul 28 16:02:14.313918 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.313650 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:14.313918 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:14.313713 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:14.314568 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314546 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-system-cni-dir\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.314628 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314585 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-etc-openvswitch\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.314628 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314609 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-log-socket\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.314788 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314657 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.314788 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314693 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05a91de5-b17a-40da-96f0-1a0a25ca6c93-system-cni-dir\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.314788 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314720 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-sysconfig\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.314788 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314742 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-sysctl-d\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.314788 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314765 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzqdt\" (UniqueName: \"kubernetes.io/projected/bbf1561e-39b1-4007-9429-aac9ae70b73c-kube-api-access-nzqdt\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.315035 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314788 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-var-lib-cni-multus\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.315035 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314818 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-run-ovn\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.315035 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314857 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-cni-bin\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.315035 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314882 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-systemd\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.315035 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314901 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-tuned\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.315035 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314919 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-run-multus-certs\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.315035 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314941 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-run-systemd\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.315035 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314963 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-var-lib-kubelet\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.315035 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.314987 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-host\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.315035 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315020 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-os-release\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315043 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-multus-daemon-config\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315069 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dswpt\" (UniqueName: \"kubernetes.io/projected/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-kube-api-access-dswpt\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315091 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-lib-modules\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315113 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs\") pod \"network-metrics-daemon-ngzg7\" (UID: \"ddb31891-1896-4c34-97c8-6e45e42d349c\") " pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315133 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d397052-5e4d-4ba0-b66a-d05699dfdebf-env-overrides\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315152 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-sys\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315173 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-cni-binary-copy\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315192 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-run-ovn-kubernetes\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315213 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-cni-netd\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315238 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk74c\" (UniqueName: \"kubernetes.io/projected/05a91de5-b17a-40da-96f0-1a0a25ca6c93-kube-api-access-nk74c\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315260 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-modprobe-d\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315280 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-run\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315308 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-var-lib-kubelet\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315329 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-node-log\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315353 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-run-netns\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315381 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffqsd\" (UniqueName: \"kubernetes.io/projected/5d397052-5e4d-4ba0-b66a-d05699dfdebf-kube-api-access-ffqsd\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.315530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315396 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-kubernetes\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315433 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/05a91de5-b17a-40da-96f0-1a0a25ca6c93-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315451 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-run-netns\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315471 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d397052-5e4d-4ba0-b66a-d05699dfdebf-ovnkube-script-lib\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315498 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-hostroot\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315519 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-kubelet\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315539 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-systemd-units\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315553 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d397052-5e4d-4ba0-b66a-d05699dfdebf-ovnkube-config\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315566 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/05a91de5-b17a-40da-96f0-1a0a25ca6c93-cni-binary-copy\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315609 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-run-k8s-cni-cncf-io\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315641 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pln4x\" (UniqueName: \"kubernetes.io/projected/ddb31891-1896-4c34-97c8-6e45e42d349c-kube-api-access-pln4x\") pod \"network-metrics-daemon-ngzg7\" (UID: \"ddb31891-1896-4c34-97c8-6e45e42d349c\") " pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315660 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/05a91de5-b17a-40da-96f0-1a0a25ca6c93-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315675 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/05a91de5-b17a-40da-96f0-1a0a25ca6c93-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315691 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-etc-kubernetes\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315709 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-run-openvswitch\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315731 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbf1561e-39b1-4007-9429-aac9ae70b73c-tmp\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.316235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315753 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-var-lib-cni-bin\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.316898 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315775 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/05a91de5-b17a-40da-96f0-1a0a25ca6c93-os-release\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.316898 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315796 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-sysctl-conf\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.316898 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315817 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-cnibin\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.316898 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315840 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-multus-cni-dir\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.316898 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315861 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-multus-socket-dir-parent\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.316898 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315883 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-multus-conf-dir\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.316898 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315917 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-slash\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.316898 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315956 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-var-lib-openvswitch\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.316898 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.315985 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d397052-5e4d-4ba0-b66a-d05699dfdebf-ovn-node-metrics-cert\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.316898 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.316007 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/05a91de5-b17a-40da-96f0-1a0a25ca6c93-cnibin\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.316898 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.316085 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:14.316898 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:14.316158 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:14.318518 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.318443 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jn6q8" Jul 28 16:02:14.321217 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.320842 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pdxnl" Jul 28 16:02:14.321217 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.320887 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-876sq\"" Jul 28 16:02:14.321217 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.320911 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 16:02:14.321217 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.321117 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 16:02:14.323049 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.323029 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 16:02:14.323153 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.323036 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 16:02:14.323153 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.323084 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dxrtv\"" Jul 28 16:02:14.323396 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.323374 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9nkv2" Jul 28 16:02:14.323592 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.323572 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 16:02:14.323681 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.323667 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 16:02:14.325459 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.325444 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bhgd6\"" Jul 28 16:02:14.325546 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.325479 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 16:02:14.325736 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.325722 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 16:02:14.358902 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.358882 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fxvlp" Jul 28 16:02:14.375934 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.375914 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fxvlp" Jul 28 16:02:14.405403 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.405380 2550 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 16:02:14.416550 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416528 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/50a3000c-4028-4746-81fa-99633b072bb5-konnectivity-ca\") pod \"konnectivity-agent-9nkv2\" (UID: \"50a3000c-4028-4746-81fa-99633b072bb5\") " pod="kube-system/konnectivity-agent-9nkv2" Jul 28 16:02:14.416632 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416565 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qv9r\" (UniqueName: \"kubernetes.io/projected/2d065aea-c380-466a-bed3-80330b2c34b1-kube-api-access-9qv9r\") pod \"iptables-alerter-zx2dx\" (UID: \"2d065aea-c380-466a-bed3-80330b2c34b1\") " pod="openshift-network-operator/iptables-alerter-zx2dx" Jul 28 16:02:14.416632 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416591 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nfrl\" (UniqueName: \"kubernetes.io/projected/9a8a71ca-e316-42e4-87c8-d64a28caab24-kube-api-access-2nfrl\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.416632 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416614 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddtmj\" (UniqueName: \"kubernetes.io/projected/088caeed-2739-4bd0-836b-3a7b0e456194-kube-api-access-ddtmj\") pod \"node-ca-pdxnl\" (UID: \"088caeed-2739-4bd0-836b-3a7b0e456194\") " pod="openshift-image-registry/node-ca-pdxnl" Jul 28 16:02:14.416774 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416642 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-run-openvswitch\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.416774 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416668 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbf1561e-39b1-4007-9429-aac9ae70b73c-tmp\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.416774 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416713 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-var-lib-cni-bin\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.416905 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416768 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-run-openvswitch\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.416905 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416818 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/05a91de5-b17a-40da-96f0-1a0a25ca6c93-os-release\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.416905 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416848 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-var-lib-cni-bin\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.416905 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416899 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-sysctl-conf\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.417097 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416933 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-cnibin\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.417097 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416929 2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 16:02:14.417097 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416975 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-multus-cni-dir\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.417097 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416994 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-cnibin\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.417097 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417001 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-multus-socket-dir-parent\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.417097 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417014 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-sysctl-conf\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.417097 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417025 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-multus-conf-dir\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.417097 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417087 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/676a8d84-5dd4-4207-91e6-a8256cdfa19e-dbus\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417112 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp4r6\" (UniqueName: \"kubernetes.io/projected/afa6eae0-4d40-4f40-9358-69feb4831e42-kube-api-access-tp4r6\") pod \"node-resolver-jn6q8\" (UID: \"afa6eae0-4d40-4f40-9358-69feb4831e42\") " pod="openshift-dns/node-resolver-jn6q8" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417129 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-multus-socket-dir-parent\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417130 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-multus-cni-dir\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417141 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-slash\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417167 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-multus-conf-dir\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417169 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-var-lib-openvswitch\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417204 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d397052-5e4d-4ba0-b66a-d05699dfdebf-ovn-node-metrics-cert\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417208 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-var-lib-openvswitch\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.416947 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/05a91de5-b17a-40da-96f0-1a0a25ca6c93-os-release\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417230 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-slash\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417243 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/05a91de5-b17a-40da-96f0-1a0a25ca6c93-cnibin\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417268 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-system-cni-dir\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417275 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/05a91de5-b17a-40da-96f0-1a0a25ca6c93-cnibin\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417305 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-etc-openvswitch\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417326 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-system-cni-dir\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417331 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-log-socket\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.417486 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417365 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-log-socket\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417361 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417382 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-etc-openvswitch\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417395 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05a91de5-b17a-40da-96f0-1a0a25ca6c93-system-cni-dir\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417448 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417455 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05a91de5-b17a-40da-96f0-1a0a25ca6c93-system-cni-dir\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417480 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-sysconfig\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417504 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-sysctl-d\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417528 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzqdt\" (UniqueName: \"kubernetes.io/projected/bbf1561e-39b1-4007-9429-aac9ae70b73c-kube-api-access-nzqdt\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417537 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-sysconfig\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417574 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-var-lib-cni-multus\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417603 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-run-ovn\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417621 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-sysctl-d\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417634 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-cni-bin\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417657 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-systemd\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417665 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-run-ovn\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417664 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-var-lib-cni-multus\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.418242 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417680 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-tuned\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417700 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-cni-bin\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417742 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-run-multus-certs\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417703 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-run-multus-certs\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417781 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/676a8d84-5dd4-4207-91e6-a8256cdfa19e-kubelet-config\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417808 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-run-systemd\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417831 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-var-lib-kubelet\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417853 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-host\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417875 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-os-release\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417901 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-multus-daemon-config\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417922 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dswpt\" (UniqueName: \"kubernetes.io/projected/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-kube-api-access-dswpt\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417945 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-registration-dir\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417963 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/088caeed-2739-4bd0-836b-3a7b0e456194-serviceca\") pod \"node-ca-pdxnl\" (UID: \"088caeed-2739-4bd0-836b-3a7b0e456194\") " pod="openshift-image-registry/node-ca-pdxnl" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417982 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-lib-modules\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418004 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs\") pod \"network-metrics-daemon-ngzg7\" (UID: \"ddb31891-1896-4c34-97c8-6e45e42d349c\") " pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418027 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418052 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/afa6eae0-4d40-4f40-9358-69feb4831e42-hosts-file\") pod \"node-resolver-jn6q8\" (UID: \"afa6eae0-4d40-4f40-9358-69feb4831e42\") " pod="openshift-dns/node-resolver-jn6q8" Jul 28 16:02:14.419063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418076 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-etc-selinux\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418099 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d397052-5e4d-4ba0-b66a-d05699dfdebf-env-overrides\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418126 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-sys\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418149 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-cni-binary-copy\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418171 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-run-ovn-kubernetes\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418196 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-cni-netd\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418220 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk74c\" (UniqueName: \"kubernetes.io/projected/05a91de5-b17a-40da-96f0-1a0a25ca6c93-kube-api-access-nk74c\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418245 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-modprobe-d\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418267 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-run\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418290 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-var-lib-kubelet\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.417753 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-systemd\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418313 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418338 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-device-dir\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418354 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-run-systemd\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418361 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-node-log\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418385 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-run-netns\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418401 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-var-lib-kubelet\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.419865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418427 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/50a3000c-4028-4746-81fa-99633b072bb5-agent-certs\") pod \"konnectivity-agent-9nkv2\" (UID: \"50a3000c-4028-4746-81fa-99633b072bb5\") " pod="kube-system/konnectivity-agent-9nkv2" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418464 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvkqd\" (UniqueName: \"kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd\") pod \"network-check-target-hxxbk\" (UID: \"833bd299-67c9-4891-9d16-cdb458c0ffa1\") " pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418469 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-host\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418491 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-socket-dir\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418514 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/088caeed-2739-4bd0-836b-3a7b0e456194-host\") pod \"node-ca-pdxnl\" (UID: \"088caeed-2739-4bd0-836b-3a7b0e456194\") " pod="openshift-image-registry/node-ca-pdxnl" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418522 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-os-release\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418548 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffqsd\" (UniqueName: \"kubernetes.io/projected/5d397052-5e4d-4ba0-b66a-d05699dfdebf-kube-api-access-ffqsd\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418636 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-lib-modules\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:14.418687 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418702 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-kubernetes\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418747 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afa6eae0-4d40-4f40-9358-69feb4831e42-tmp-dir\") pod \"node-resolver-jn6q8\" (UID: \"afa6eae0-4d40-4f40-9358-69feb4831e42\") " pod="openshift-dns/node-resolver-jn6q8" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418781 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/05a91de5-b17a-40da-96f0-1a0a25ca6c93-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418808 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-run-netns\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418837 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d397052-5e4d-4ba0-b66a-d05699dfdebf-ovnkube-script-lib\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418860 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-hostroot\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418897 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-sys-fs\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418922 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-kubelet\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418945 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-systemd-units\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.420833 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418968 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d397052-5e4d-4ba0-b66a-d05699dfdebf-ovnkube-config\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.418993 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/05a91de5-b17a-40da-96f0-1a0a25ca6c93-cni-binary-copy\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419020 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-run-k8s-cni-cncf-io\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419046 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pln4x\" (UniqueName: \"kubernetes.io/projected/ddb31891-1896-4c34-97c8-6e45e42d349c-kube-api-access-pln4x\") pod \"network-metrics-daemon-ngzg7\" (UID: \"ddb31891-1896-4c34-97c8-6e45e42d349c\") " pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419073 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d065aea-c380-466a-bed3-80330b2c34b1-iptables-alerter-script\") pod \"iptables-alerter-zx2dx\" (UID: \"2d065aea-c380-466a-bed3-80330b2c34b1\") " pod="openshift-network-operator/iptables-alerter-zx2dx" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419086 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-multus-daemon-config\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419153 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-kubernetes\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419169 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-sys\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419191 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d065aea-c380-466a-bed3-80330b2c34b1-host-slash\") pod \"iptables-alerter-zx2dx\" (UID: \"2d065aea-c380-466a-bed3-80330b2c34b1\") " pod="openshift-network-operator/iptables-alerter-zx2dx" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419228 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/05a91de5-b17a-40da-96f0-1a0a25ca6c93-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419255 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/05a91de5-b17a-40da-96f0-1a0a25ca6c93-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419263 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d397052-5e4d-4ba0-b66a-d05699dfdebf-env-overrides\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419279 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-etc-kubernetes\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419306 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-node-log\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419340 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-run-netns\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419373 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-etc-kubernetes\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419464 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/05a91de5-b17a-40da-96f0-1a0a25ca6c93-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.421480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419515 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-run-netns\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:14.419636 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs podName:ddb31891-1896-4c34-97c8-6e45e42d349c nodeName:}" failed. No retries permitted until 2026-07-28 16:02:14.919615335 +0000 UTC m=+2.953782659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs") pod "network-metrics-daemon-ngzg7" (UID: "ddb31891-1896-4c34-97c8-6e45e42d349c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419636 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-cni-binary-copy\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419672 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-run-ovn-kubernetes\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419691 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d397052-5e4d-4ba0-b66a-d05699dfdebf-ovnkube-config\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419711 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-cni-netd\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419197 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-host-kubelet\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419230 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d397052-5e4d-4ba0-b66a-d05699dfdebf-systemd-units\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419890 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-run\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.419970 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-modprobe-d\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.420015 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-var-lib-kubelet\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.420033 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d397052-5e4d-4ba0-b66a-d05699dfdebf-ovnkube-script-lib\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.420065 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-host-run-k8s-cni-cncf-io\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.420100 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-hostroot\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.420264 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/05a91de5-b17a-40da-96f0-1a0a25ca6c93-cni-binary-copy\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.420309 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/05a91de5-b17a-40da-96f0-1a0a25ca6c93-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.420571 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/05a91de5-b17a-40da-96f0-1a0a25ca6c93-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.422159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.420727 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbf1561e-39b1-4007-9429-aac9ae70b73c-tmp\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.422894 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.420819 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d397052-5e4d-4ba0-b66a-d05699dfdebf-ovn-node-metrics-cert\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.422894 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.422188 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbf1561e-39b1-4007-9429-aac9ae70b73c-etc-tuned\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.430627 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.430600 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzqdt\" (UniqueName: \"kubernetes.io/projected/bbf1561e-39b1-4007-9429-aac9ae70b73c-kube-api-access-nzqdt\") pod \"tuned-8xx66\" (UID: \"bbf1561e-39b1-4007-9429-aac9ae70b73c\") " pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.433816 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.433751 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk74c\" (UniqueName: \"kubernetes.io/projected/05a91de5-b17a-40da-96f0-1a0a25ca6c93-kube-api-access-nk74c\") pod \"multus-additional-cni-plugins-2ldwp\" (UID: \"05a91de5-b17a-40da-96f0-1a0a25ca6c93\") " pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.433816 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.433788 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffqsd\" (UniqueName: \"kubernetes.io/projected/5d397052-5e4d-4ba0-b66a-d05699dfdebf-kube-api-access-ffqsd\") pod \"ovnkube-node-4xnwq\" (UID: \"5d397052-5e4d-4ba0-b66a-d05699dfdebf\") " pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.433973 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.433856 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pln4x\" (UniqueName: \"kubernetes.io/projected/ddb31891-1896-4c34-97c8-6e45e42d349c-kube-api-access-pln4x\") pod \"network-metrics-daemon-ngzg7\" (UID: \"ddb31891-1896-4c34-97c8-6e45e42d349c\") " pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:14.435317 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.435299 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dswpt\" (UniqueName: \"kubernetes.io/projected/b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb-kube-api-access-dswpt\") pod \"multus-88478\" (UID: \"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb\") " pod="openshift-multus/multus-88478" Jul 28 16:02:14.440674 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.440620 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-10.ec2.internal" event={"ID":"ce1d00f21bc867d7fa218ae3a0c694e1","Type":"ContainerStarted","Data":"20e9c7e7a7ae3b00e74d5f460bbac807fc9917bc6342fc8d8d1757ed929dba9c"} Jul 28 16:02:14.441858 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.441802 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal" event={"ID":"0d72e1a34529c7526e7d432be101f4be","Type":"ContainerStarted","Data":"b097564cc36a36c51e01d5db9c84ff7f1533fb5aee2a9fbf9426b7768685cb76"} Jul 28 16:02:14.519585 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.519553 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d065aea-c380-466a-bed3-80330b2c34b1-iptables-alerter-script\") pod \"iptables-alerter-zx2dx\" (UID: \"2d065aea-c380-466a-bed3-80330b2c34b1\") " pod="openshift-network-operator/iptables-alerter-zx2dx" Jul 28 16:02:14.519713 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.519591 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d065aea-c380-466a-bed3-80330b2c34b1-host-slash\") pod \"iptables-alerter-zx2dx\" (UID: \"2d065aea-c380-466a-bed3-80330b2c34b1\") " pod="openshift-network-operator/iptables-alerter-zx2dx" Jul 28 16:02:14.519713 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.519624 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/50a3000c-4028-4746-81fa-99633b072bb5-konnectivity-ca\") pod \"konnectivity-agent-9nkv2\" (UID: \"50a3000c-4028-4746-81fa-99633b072bb5\") " pod="kube-system/konnectivity-agent-9nkv2" Jul 28 16:02:14.519713 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.519683 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d065aea-c380-466a-bed3-80330b2c34b1-host-slash\") pod \"iptables-alerter-zx2dx\" (UID: \"2d065aea-c380-466a-bed3-80330b2c34b1\") " pod="openshift-network-operator/iptables-alerter-zx2dx" Jul 28 16:02:14.519881 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.519716 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qv9r\" (UniqueName: \"kubernetes.io/projected/2d065aea-c380-466a-bed3-80330b2c34b1-kube-api-access-9qv9r\") pod \"iptables-alerter-zx2dx\" (UID: \"2d065aea-c380-466a-bed3-80330b2c34b1\") " pod="openshift-network-operator/iptables-alerter-zx2dx" Jul 28 16:02:14.519881 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.519766 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nfrl\" (UniqueName: \"kubernetes.io/projected/9a8a71ca-e316-42e4-87c8-d64a28caab24-kube-api-access-2nfrl\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.519881 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.519794 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddtmj\" (UniqueName: \"kubernetes.io/projected/088caeed-2739-4bd0-836b-3a7b0e456194-kube-api-access-ddtmj\") pod \"node-ca-pdxnl\" (UID: \"088caeed-2739-4bd0-836b-3a7b0e456194\") " pod="openshift-image-registry/node-ca-pdxnl" Jul 28 16:02:14.519881 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.519825 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/676a8d84-5dd4-4207-91e6-a8256cdfa19e-dbus\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:14.519881 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.519850 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp4r6\" (UniqueName: \"kubernetes.io/projected/afa6eae0-4d40-4f40-9358-69feb4831e42-kube-api-access-tp4r6\") pod \"node-resolver-jn6q8\" (UID: \"afa6eae0-4d40-4f40-9358-69feb4831e42\") " pod="openshift-dns/node-resolver-jn6q8" Jul 28 16:02:14.520111 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.519897 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/676a8d84-5dd4-4207-91e6-a8256cdfa19e-kubelet-config\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:14.520111 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.519926 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-registration-dir\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.520111 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.519993 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/676a8d84-5dd4-4207-91e6-a8256cdfa19e-dbus\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:14.520111 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520050 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-registration-dir\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.520297 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520139 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/50a3000c-4028-4746-81fa-99633b072bb5-konnectivity-ca\") pod \"konnectivity-agent-9nkv2\" (UID: \"50a3000c-4028-4746-81fa-99633b072bb5\") " pod="kube-system/konnectivity-agent-9nkv2" Jul 28 16:02:14.520297 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520158 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d065aea-c380-466a-bed3-80330b2c34b1-iptables-alerter-script\") pod \"iptables-alerter-zx2dx\" (UID: \"2d065aea-c380-466a-bed3-80330b2c34b1\") " pod="openshift-network-operator/iptables-alerter-zx2dx" Jul 28 16:02:14.520297 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520182 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/676a8d84-5dd4-4207-91e6-a8256cdfa19e-kubelet-config\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:14.520297 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520198 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/088caeed-2739-4bd0-836b-3a7b0e456194-serviceca\") pod \"node-ca-pdxnl\" (UID: \"088caeed-2739-4bd0-836b-3a7b0e456194\") " pod="openshift-image-registry/node-ca-pdxnl" Jul 28 16:02:14.520297 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520242 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:14.520297 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520267 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/afa6eae0-4d40-4f40-9358-69feb4831e42-hosts-file\") pod \"node-resolver-jn6q8\" (UID: \"afa6eae0-4d40-4f40-9358-69feb4831e42\") " pod="openshift-dns/node-resolver-jn6q8" Jul 28 16:02:14.520589 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520317 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/afa6eae0-4d40-4f40-9358-69feb4831e42-hosts-file\") pod \"node-resolver-jn6q8\" (UID: \"afa6eae0-4d40-4f40-9358-69feb4831e42\") " pod="openshift-dns/node-resolver-jn6q8" Jul 28 16:02:14.520589 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:14.520347 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:02:14.520589 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:14.520418 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret podName:676a8d84-5dd4-4207-91e6-a8256cdfa19e nodeName:}" failed. No retries permitted until 2026-07-28 16:02:15.020384725 +0000 UTC m=+3.054552031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret") pod "global-pull-secret-syncer-nww4s" (UID: "676a8d84-5dd4-4207-91e6-a8256cdfa19e") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:02:14.520589 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520439 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-etc-selinux\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.520589 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520475 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.520589 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520497 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-device-dir\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.520589 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520525 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/50a3000c-4028-4746-81fa-99633b072bb5-agent-certs\") pod \"konnectivity-agent-9nkv2\" (UID: \"50a3000c-4028-4746-81fa-99633b072bb5\") " pod="kube-system/konnectivity-agent-9nkv2" Jul 28 16:02:14.520589 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520552 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvkqd\" (UniqueName: \"kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd\") pod \"network-check-target-hxxbk\" (UID: \"833bd299-67c9-4891-9d16-cdb458c0ffa1\") " pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:14.520589 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520563 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/088caeed-2739-4bd0-836b-3a7b0e456194-serviceca\") pod \"node-ca-pdxnl\" (UID: \"088caeed-2739-4bd0-836b-3a7b0e456194\") " pod="openshift-image-registry/node-ca-pdxnl" Jul 28 16:02:14.520589 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520577 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-socket-dir\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.521061 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520598 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/088caeed-2739-4bd0-836b-3a7b0e456194-host\") pod \"node-ca-pdxnl\" (UID: \"088caeed-2739-4bd0-836b-3a7b0e456194\") " pod="openshift-image-registry/node-ca-pdxnl" Jul 28 16:02:14.521061 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520604 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.521061 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520604 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-etc-selinux\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.521061 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520623 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afa6eae0-4d40-4f40-9358-69feb4831e42-tmp-dir\") pod \"node-resolver-jn6q8\" (UID: \"afa6eae0-4d40-4f40-9358-69feb4831e42\") " pod="openshift-dns/node-resolver-jn6q8" Jul 28 16:02:14.521061 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520652 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-sys-fs\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.521061 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520660 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-socket-dir\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.521061 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520601 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-device-dir\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.521061 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520717 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9a8a71ca-e316-42e4-87c8-d64a28caab24-sys-fs\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.521061 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520747 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/088caeed-2739-4bd0-836b-3a7b0e456194-host\") pod \"node-ca-pdxnl\" (UID: \"088caeed-2739-4bd0-836b-3a7b0e456194\") " pod="openshift-image-registry/node-ca-pdxnl" Jul 28 16:02:14.521061 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.520925 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afa6eae0-4d40-4f40-9358-69feb4831e42-tmp-dir\") pod \"node-resolver-jn6q8\" (UID: \"afa6eae0-4d40-4f40-9358-69feb4831e42\") " pod="openshift-dns/node-resolver-jn6q8" Jul 28 16:02:14.522907 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.522885 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/50a3000c-4028-4746-81fa-99633b072bb5-agent-certs\") pod \"konnectivity-agent-9nkv2\" (UID: \"50a3000c-4028-4746-81fa-99633b072bb5\") " pod="kube-system/konnectivity-agent-9nkv2" Jul 28 16:02:14.527235 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:14.527215 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:02:14.527235 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:14.527236 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:02:14.527401 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:14.527250 2550 projected.go:196] Error preparing data for projected volume kube-api-access-fvkqd for pod openshift-network-diagnostics/network-check-target-hxxbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:02:14.527401 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:14.527302 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd podName:833bd299-67c9-4891-9d16-cdb458c0ffa1 nodeName:}" failed. No retries permitted until 2026-07-28 16:02:15.027286183 +0000 UTC m=+3.061453492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fvkqd" (UniqueName: "kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd") pod "network-check-target-hxxbk" (UID: "833bd299-67c9-4891-9d16-cdb458c0ffa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:02:14.529082 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.529051 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qv9r\" (UniqueName: \"kubernetes.io/projected/2d065aea-c380-466a-bed3-80330b2c34b1-kube-api-access-9qv9r\") pod \"iptables-alerter-zx2dx\" (UID: \"2d065aea-c380-466a-bed3-80330b2c34b1\") " pod="openshift-network-operator/iptables-alerter-zx2dx" Jul 28 16:02:14.529197 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.529060 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddtmj\" (UniqueName: \"kubernetes.io/projected/088caeed-2739-4bd0-836b-3a7b0e456194-kube-api-access-ddtmj\") pod \"node-ca-pdxnl\" (UID: \"088caeed-2739-4bd0-836b-3a7b0e456194\") " pod="openshift-image-registry/node-ca-pdxnl" Jul 28 16:02:14.531246 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.531222 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nfrl\" (UniqueName: \"kubernetes.io/projected/9a8a71ca-e316-42e4-87c8-d64a28caab24-kube-api-access-2nfrl\") pod \"aws-ebs-csi-driver-node-pk5kg\" (UID: \"9a8a71ca-e316-42e4-87c8-d64a28caab24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.531353 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.531270 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp4r6\" (UniqueName: \"kubernetes.io/projected/afa6eae0-4d40-4f40-9358-69feb4831e42-kube-api-access-tp4r6\") pod \"node-resolver-jn6q8\" (UID: \"afa6eae0-4d40-4f40-9358-69feb4831e42\") " pod="openshift-dns/node-resolver-jn6q8" Jul 28 16:02:14.608119 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.608096 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:14.615261 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.615244 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2ldwp" Jul 28 16:02:14.625848 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.625830 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8xx66" Jul 28 16:02:14.630377 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.630356 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-88478" Jul 28 16:02:14.636972 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.636955 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zx2dx" Jul 28 16:02:14.645546 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.645528 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" Jul 28 16:02:14.645647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.645626 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:02:14.660017 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.659994 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jn6q8" Jul 28 16:02:14.667625 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.667604 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pdxnl" Jul 28 16:02:14.673095 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.673079 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9nkv2" Jul 28 16:02:14.923871 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:14.923799 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs\") pod \"network-metrics-daemon-ngzg7\" (UID: \"ddb31891-1896-4c34-97c8-6e45e42d349c\") " pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:14.924021 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:14.923930 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:02:14.924021 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:14.924000 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs podName:ddb31891-1896-4c34-97c8-6e45e42d349c nodeName:}" failed. No retries permitted until 2026-07-28 16:02:15.923982008 +0000 UTC m=+3.958149328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs") pod "network-metrics-daemon-ngzg7" (UID: "ddb31891-1896-4c34-97c8-6e45e42d349c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:02:15.024717 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.024684 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:15.024877 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:15.024806 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:02:15.024930 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:15.024882 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret podName:676a8d84-5dd4-4207-91e6-a8256cdfa19e nodeName:}" failed. No retries permitted until 2026-07-28 16:02:16.024862469 +0000 UTC m=+4.059029801 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret") pod "global-pull-secret-syncer-nww4s" (UID: "676a8d84-5dd4-4207-91e6-a8256cdfa19e") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:02:15.066665 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:15.066639 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbf1561e_39b1_4007_9429_aac9ae70b73c.slice/crio-5296c9c96b6b425c198d5050b8e3cb0e163a5d7f2c879ca724f3b5f877f79e6b WatchSource:0}: Error finding container 5296c9c96b6b425c198d5050b8e3cb0e163a5d7f2c879ca724f3b5f877f79e6b: Status 404 returned error can't find the container with id 5296c9c96b6b425c198d5050b8e3cb0e163a5d7f2c879ca724f3b5f877f79e6b Jul 28 16:02:15.067258 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:15.067230 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a8a71ca_e316_42e4_87c8_d64a28caab24.slice/crio-aaba5ad37f353b550ded5d9ed830e476376edadc97713ef1a81bc716a70ec3a4 WatchSource:0}: Error finding container aaba5ad37f353b550ded5d9ed830e476376edadc97713ef1a81bc716a70ec3a4: Status 404 returned error can't find the container with id aaba5ad37f353b550ded5d9ed830e476376edadc97713ef1a81bc716a70ec3a4 Jul 28 16:02:15.071278 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:15.071257 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafa6eae0_4d40_4f40_9358_69feb4831e42.slice/crio-1471fefd75852880480b5544cead5dd93f417323b611586529232aba6b63df04 WatchSource:0}: Error finding container 1471fefd75852880480b5544cead5dd93f417323b611586529232aba6b63df04: Status 404 returned error can't find the container with id 1471fefd75852880480b5544cead5dd93f417323b611586529232aba6b63df04 Jul 28 16:02:15.071994 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:15.071973 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb46c2d3e_bf32_4f6a_ab84_6e84d1a6abbb.slice/crio-a455bc1ccdad6b154629310334a4d5207a92dc853990f11e0a2f5565cd4451d0 WatchSource:0}: Error finding container a455bc1ccdad6b154629310334a4d5207a92dc853990f11e0a2f5565cd4451d0: Status 404 returned error can't find the container with id a455bc1ccdad6b154629310334a4d5207a92dc853990f11e0a2f5565cd4451d0 Jul 28 16:02:15.093898 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:15.093871 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d397052_5e4d_4ba0_b66a_d05699dfdebf.slice/crio-bbd8e99e7f96adad0a935db359047a95d79a307bf55fe9b1ac607f321e8b1661 WatchSource:0}: Error finding container bbd8e99e7f96adad0a935db359047a95d79a307bf55fe9b1ac607f321e8b1661: Status 404 returned error can't find the container with id bbd8e99e7f96adad0a935db359047a95d79a307bf55fe9b1ac607f321e8b1661 Jul 28 16:02:15.094930 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:15.094913 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50a3000c_4028_4746_81fa_99633b072bb5.slice/crio-fe55f27c974a3c157e91c106d0c37567c99431616d1f9b1133dc3cba6ba3f062 WatchSource:0}: Error finding container fe55f27c974a3c157e91c106d0c37567c99431616d1f9b1133dc3cba6ba3f062: Status 404 returned error can't find the container with id fe55f27c974a3c157e91c106d0c37567c99431616d1f9b1133dc3cba6ba3f062 Jul 28 16:02:15.124962 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.124930 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvkqd\" (UniqueName: \"kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd\") pod \"network-check-target-hxxbk\" (UID: \"833bd299-67c9-4891-9d16-cdb458c0ffa1\") " pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:15.125087 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:15.125073 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:02:15.125133 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:15.125093 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:02:15.125133 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:15.125103 2550 projected.go:196] Error preparing data for projected volume kube-api-access-fvkqd for pod openshift-network-diagnostics/network-check-target-hxxbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:02:15.125195 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:15.125146 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd podName:833bd299-67c9-4891-9d16-cdb458c0ffa1 nodeName:}" failed. No retries permitted until 2026-07-28 16:02:16.12513068 +0000 UTC m=+4.159297983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fvkqd" (UniqueName: "kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd") pod "network-check-target-hxxbk" (UID: "833bd299-67c9-4891-9d16-cdb458c0ffa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:02:15.381394 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.381362 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 15:57:14 +0000 UTC" deadline="2028-01-24 03:42:16.313438526 +0000 UTC" Jul 28 16:02:15.381394 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.381389 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13067h40m0.932053036s" Jul 28 16:02:15.444341 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.444313 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-88478" event={"ID":"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb","Type":"ContainerStarted","Data":"a455bc1ccdad6b154629310334a4d5207a92dc853990f11e0a2f5565cd4451d0"} Jul 28 16:02:15.445368 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.445350 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jn6q8" event={"ID":"afa6eae0-4d40-4f40-9358-69feb4831e42","Type":"ContainerStarted","Data":"1471fefd75852880480b5544cead5dd93f417323b611586529232aba6b63df04"} Jul 28 16:02:15.446423 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.446386 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9nkv2" event={"ID":"50a3000c-4028-4746-81fa-99633b072bb5","Type":"ContainerStarted","Data":"fe55f27c974a3c157e91c106d0c37567c99431616d1f9b1133dc3cba6ba3f062"} Jul 28 16:02:15.447339 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.447320 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" event={"ID":"9a8a71ca-e316-42e4-87c8-d64a28caab24","Type":"ContainerStarted","Data":"aaba5ad37f353b550ded5d9ed830e476376edadc97713ef1a81bc716a70ec3a4"} Jul 28 16:02:15.448245 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.448227 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8xx66" event={"ID":"bbf1561e-39b1-4007-9429-aac9ae70b73c","Type":"ContainerStarted","Data":"5296c9c96b6b425c198d5050b8e3cb0e163a5d7f2c879ca724f3b5f877f79e6b"} Jul 28 16:02:15.449805 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.449786 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-10.ec2.internal" event={"ID":"ce1d00f21bc867d7fa218ae3a0c694e1","Type":"ContainerStarted","Data":"5956606bfdfdd166edc3f08df2a774baeb3b507dc8ab5c459ac79be320829251"} Jul 28 16:02:15.450882 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.450858 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" event={"ID":"5d397052-5e4d-4ba0-b66a-d05699dfdebf","Type":"ContainerStarted","Data":"bbd8e99e7f96adad0a935db359047a95d79a307bf55fe9b1ac607f321e8b1661"} Jul 28 16:02:15.451797 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.451778 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zx2dx" event={"ID":"2d065aea-c380-466a-bed3-80330b2c34b1","Type":"ContainerStarted","Data":"0c73363ba30dec484ea3f18d073d74113491b2b14eb5f8a36704073c51d424e4"} Jul 28 16:02:15.452713 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.452690 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pdxnl" event={"ID":"088caeed-2739-4bd0-836b-3a7b0e456194","Type":"ContainerStarted","Data":"e7eedfcfda6a36562911dc8d63f5013b35a2498ec40d210d530f08654109520c"} Jul 28 16:02:15.453781 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.453764 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2ldwp" event={"ID":"05a91de5-b17a-40da-96f0-1a0a25ca6c93","Type":"ContainerStarted","Data":"b69909ef04ac8eb2b3b4a4cc92465695968abaca058ac9c5a0aa9e4d0bc034d5"} Jul 28 16:02:15.467552 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.466396 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-10.ec2.internal" podStartSLOduration=2.466373931 podStartE2EDuration="2.466373931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:02:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:15.465501325 +0000 UTC m=+3.499668652" watchObservedRunningTime="2026-07-28 16:02:15.466373931 +0000 UTC m=+3.500541258" Jul 28 16:02:15.932856 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:15.932817 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs\") pod \"network-metrics-daemon-ngzg7\" (UID: \"ddb31891-1896-4c34-97c8-6e45e42d349c\") " pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:15.933048 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:15.932964 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:02:15.933048 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:15.933028 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs podName:ddb31891-1896-4c34-97c8-6e45e42d349c nodeName:}" failed. No retries permitted until 2026-07-28 16:02:17.933010032 +0000 UTC m=+5.967177340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs") pod "network-metrics-daemon-ngzg7" (UID: "ddb31891-1896-4c34-97c8-6e45e42d349c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:02:16.033669 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:16.033630 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:16.033853 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:16.033834 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:02:16.033921 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:16.033911 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret podName:676a8d84-5dd4-4207-91e6-a8256cdfa19e nodeName:}" failed. No retries permitted until 2026-07-28 16:02:18.033892468 +0000 UTC m=+6.068059774 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret") pod "global-pull-secret-syncer-nww4s" (UID: "676a8d84-5dd4-4207-91e6-a8256cdfa19e") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:02:16.134158 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:16.133991 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvkqd\" (UniqueName: \"kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd\") pod \"network-check-target-hxxbk\" (UID: \"833bd299-67c9-4891-9d16-cdb458c0ffa1\") " pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:16.134306 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:16.134170 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:02:16.134306 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:16.134195 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:02:16.134306 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:16.134207 2550 projected.go:196] Error preparing data for projected volume kube-api-access-fvkqd for pod openshift-network-diagnostics/network-check-target-hxxbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:02:16.134306 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:16.134262 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd podName:833bd299-67c9-4891-9d16-cdb458c0ffa1 nodeName:}" failed. No retries permitted until 2026-07-28 16:02:18.134243553 +0000 UTC m=+6.168410862 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fvkqd" (UniqueName: "kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd") pod "network-check-target-hxxbk" (UID: "833bd299-67c9-4891-9d16-cdb458c0ffa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:02:16.382074 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:16.382006 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 15:57:14 +0000 UTC" deadline="2028-01-14 03:01:48.519324249 +0000 UTC" Jul 28 16:02:16.382074 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:16.382071 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12826h59m32.137257575s" Jul 28 16:02:16.437279 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:16.436505 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:16.437279 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:16.436618 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:16.437279 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:16.436987 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:16.437279 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:16.437085 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:16.437279 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:16.437151 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:16.437279 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:16.437243 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:17.946750 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:17.946503 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs\") pod \"network-metrics-daemon-ngzg7\" (UID: \"ddb31891-1896-4c34-97c8-6e45e42d349c\") " pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:17.947217 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:17.946922 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:02:17.947217 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:17.946985 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs podName:ddb31891-1896-4c34-97c8-6e45e42d349c nodeName:}" failed. No retries permitted until 2026-07-28 16:02:21.946964258 +0000 UTC m=+9.981131574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs") pod "network-metrics-daemon-ngzg7" (UID: "ddb31891-1896-4c34-97c8-6e45e42d349c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:02:18.047735 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:18.047694 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:18.047955 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:18.047848 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:02:18.047955 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:18.047921 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret podName:676a8d84-5dd4-4207-91e6-a8256cdfa19e nodeName:}" failed. No retries permitted until 2026-07-28 16:02:22.047900032 +0000 UTC m=+10.082067340 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret") pod "global-pull-secret-syncer-nww4s" (UID: "676a8d84-5dd4-4207-91e6-a8256cdfa19e") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:02:18.149646 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:18.148927 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvkqd\" (UniqueName: \"kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd\") pod \"network-check-target-hxxbk\" (UID: \"833bd299-67c9-4891-9d16-cdb458c0ffa1\") " pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:18.149646 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:18.149176 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:02:18.149646 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:18.149201 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:02:18.149646 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:18.149213 2550 projected.go:196] Error preparing data for projected volume kube-api-access-fvkqd for pod openshift-network-diagnostics/network-check-target-hxxbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:02:18.149646 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:18.149266 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd podName:833bd299-67c9-4891-9d16-cdb458c0ffa1 nodeName:}" failed. No retries permitted until 2026-07-28 16:02:22.149249523 +0000 UTC m=+10.183416833 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fvkqd" (UniqueName: "kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd") pod "network-check-target-hxxbk" (UID: "833bd299-67c9-4891-9d16-cdb458c0ffa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:02:18.435966 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:18.435937 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:18.436158 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:18.436065 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:18.436217 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:18.436137 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:18.436342 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:18.436272 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:18.436342 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:18.436342 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:18.436542 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:18.436455 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:20.435867 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:20.435833 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:20.436368 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:20.435964 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:20.436446 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:20.436376 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:20.436446 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:20.436398 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:20.436585 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:20.436543 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:20.436637 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:20.436615 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:21.982185 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:21.981746 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs\") pod \"network-metrics-daemon-ngzg7\" (UID: \"ddb31891-1896-4c34-97c8-6e45e42d349c\") " pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:21.982185 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:21.981878 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:02:21.982185 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:21.981926 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs podName:ddb31891-1896-4c34-97c8-6e45e42d349c nodeName:}" failed. No retries permitted until 2026-07-28 16:02:29.981914139 +0000 UTC m=+18.016081443 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs") pod "network-metrics-daemon-ngzg7" (UID: "ddb31891-1896-4c34-97c8-6e45e42d349c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:02:22.082689 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:22.082588 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:22.082839 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:22.082751 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:02:22.082839 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:22.082819 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret podName:676a8d84-5dd4-4207-91e6-a8256cdfa19e nodeName:}" failed. No retries permitted until 2026-07-28 16:02:30.082798458 +0000 UTC m=+18.116965764 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret") pod "global-pull-secret-syncer-nww4s" (UID: "676a8d84-5dd4-4207-91e6-a8256cdfa19e") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:02:22.183200 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:22.183167 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvkqd\" (UniqueName: \"kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd\") pod \"network-check-target-hxxbk\" (UID: \"833bd299-67c9-4891-9d16-cdb458c0ffa1\") " pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:22.183375 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:22.183346 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:02:22.183375 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:22.183372 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:02:22.183564 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:22.183386 2550 projected.go:196] Error preparing data for projected volume kube-api-access-fvkqd for pod openshift-network-diagnostics/network-check-target-hxxbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:02:22.183564 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:22.183459 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd podName:833bd299-67c9-4891-9d16-cdb458c0ffa1 nodeName:}" failed. No retries permitted until 2026-07-28 16:02:30.183439599 +0000 UTC m=+18.217606925 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fvkqd" (UniqueName: "kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd") pod "network-check-target-hxxbk" (UID: "833bd299-67c9-4891-9d16-cdb458c0ffa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:02:22.437444 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:22.437213 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:22.437444 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:22.437292 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:22.437444 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:22.437336 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:22.437444 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:22.437393 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:22.437782 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:22.437592 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:22.437926 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:22.437886 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:24.435745 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:24.435712 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:24.436257 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:24.435712 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:24.436257 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:24.435855 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:24.436257 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:24.435724 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:24.436257 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:24.435915 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:24.436257 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:24.436019 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:26.435721 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:26.435686 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:26.436176 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:26.435700 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:26.436176 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:26.435805 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:26.436176 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:26.435834 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:26.436176 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:26.435950 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:26.436176 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:26.436048 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:28.435642 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:28.435610 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:28.436092 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:28.435609 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:28.436092 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:28.435729 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:28.436092 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:28.435744 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:28.436092 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:28.435823 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:28.436092 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:28.435914 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:30.047147 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:30.047113 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs\") pod \"network-metrics-daemon-ngzg7\" (UID: \"ddb31891-1896-4c34-97c8-6e45e42d349c\") " pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:30.047630 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:30.047246 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:02:30.047630 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:30.047308 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs podName:ddb31891-1896-4c34-97c8-6e45e42d349c nodeName:}" failed. No retries permitted until 2026-07-28 16:02:46.047288468 +0000 UTC m=+34.081455772 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs") pod "network-metrics-daemon-ngzg7" (UID: "ddb31891-1896-4c34-97c8-6e45e42d349c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:02:30.148276 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:30.148244 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:30.148440 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:30.148396 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:02:30.148500 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:30.148469 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret podName:676a8d84-5dd4-4207-91e6-a8256cdfa19e nodeName:}" failed. No retries permitted until 2026-07-28 16:02:46.148454648 +0000 UTC m=+34.182621951 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret") pod "global-pull-secret-syncer-nww4s" (UID: "676a8d84-5dd4-4207-91e6-a8256cdfa19e") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:02:30.248753 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:30.248726 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvkqd\" (UniqueName: \"kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd\") pod \"network-check-target-hxxbk\" (UID: \"833bd299-67c9-4891-9d16-cdb458c0ffa1\") " pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:30.248905 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:30.248844 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:02:30.248905 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:30.248861 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:02:30.248905 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:30.248870 2550 projected.go:196] Error preparing data for projected volume kube-api-access-fvkqd for pod openshift-network-diagnostics/network-check-target-hxxbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:02:30.249038 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:30.248912 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd podName:833bd299-67c9-4891-9d16-cdb458c0ffa1 nodeName:}" failed. No retries permitted until 2026-07-28 16:02:46.248899527 +0000 UTC m=+34.283066831 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fvkqd" (UniqueName: "kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd") pod "network-check-target-hxxbk" (UID: "833bd299-67c9-4891-9d16-cdb458c0ffa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:02:30.436528 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:30.436496 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:30.436528 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:30.436512 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:30.436732 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:30.436496 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:30.436732 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:30.436621 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:30.436732 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:30.436687 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:30.436906 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:30.436778 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:32.436647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:32.436615 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:32.437078 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:32.436717 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:32.437078 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:32.436770 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:32.437078 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:32.436838 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:32.437078 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:32.436875 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:32.437078 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:32.436973 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:34.436500 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.436261 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:34.437200 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.436261 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:34.437200 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:34.436597 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:34.437200 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.436337 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:34.437200 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:34.436672 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:34.437200 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:34.436782 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:34.494102 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.494077 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pdxnl" event={"ID":"088caeed-2739-4bd0-836b-3a7b0e456194","Type":"ContainerStarted","Data":"f9cd52536e0e293e1ed81c3c9bedc60fcc0b9680635411c6fc89dd70f695a576"} Jul 28 16:02:34.495463 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.495443 2550 generic.go:356] "Generic (PLEG): container finished" podID="05a91de5-b17a-40da-96f0-1a0a25ca6c93" containerID="5cb3b102e4a297edf9bb878bb27717537bca8d2d9dfc48b94428636f352c0b40" exitCode=0 Jul 28 16:02:34.495532 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.495511 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2ldwp" event={"ID":"05a91de5-b17a-40da-96f0-1a0a25ca6c93","Type":"ContainerDied","Data":"5cb3b102e4a297edf9bb878bb27717537bca8d2d9dfc48b94428636f352c0b40"} Jul 28 16:02:34.496879 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.496851 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-88478" event={"ID":"b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb","Type":"ContainerStarted","Data":"cafed3d881e69b29068166c36df640f0b8c705efd0e5903e47bed1dd9d5cd44b"} Jul 28 16:02:34.498259 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.498222 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jn6q8" event={"ID":"afa6eae0-4d40-4f40-9358-69feb4831e42","Type":"ContainerStarted","Data":"6da8baae5140393bba79b4c01736e195280fbc656e7b0be4996bc3047f55d286"} Jul 28 16:02:34.499520 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.499499 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9nkv2" event={"ID":"50a3000c-4028-4746-81fa-99633b072bb5","Type":"ContainerStarted","Data":"fea56978f5477f518bd65decb71ac1e04ab7bacda3bfbc878e2efd5ee3c2d02b"} Jul 28 16:02:34.500869 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.500851 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" event={"ID":"9a8a71ca-e316-42e4-87c8-d64a28caab24","Type":"ContainerStarted","Data":"f772c1e87aed680cf02dc0e8a86fa648117a6a8e82749430a677425440a378a0"} Jul 28 16:02:34.502241 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.502222 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8xx66" event={"ID":"bbf1561e-39b1-4007-9429-aac9ae70b73c","Type":"ContainerStarted","Data":"5dfe91b63eca89c94ffbcd29e4e42b198e04536adbf8f4a2a2d564aa3bcd2e23"} Jul 28 16:02:34.503512 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.503491 2550 generic.go:356] "Generic (PLEG): container finished" podID="0d72e1a34529c7526e7d432be101f4be" containerID="c56fe4795c6bb80312085892333ee774c9c0d8c8f9f15021e13d38efce53edd8" exitCode=0 Jul 28 16:02:34.503598 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.503577 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal" event={"ID":"0d72e1a34529c7526e7d432be101f4be","Type":"ContainerDied","Data":"c56fe4795c6bb80312085892333ee774c9c0d8c8f9f15021e13d38efce53edd8"} Jul 28 16:02:34.509307 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.509288 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" event={"ID":"5d397052-5e4d-4ba0-b66a-d05699dfdebf","Type":"ContainerStarted","Data":"8bbd40f373e7be42ad5082bcb76822c2bcd9f1a93ff39f9092e85eea8daf50dd"} Jul 28 16:02:34.509399 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.509385 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" event={"ID":"5d397052-5e4d-4ba0-b66a-d05699dfdebf","Type":"ContainerStarted","Data":"c81b6d82ae85e70e663b5d9d45c55259c7301aeccccae425d728e8d40895857c"} Jul 28 16:02:34.509477 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.509438 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" event={"ID":"5d397052-5e4d-4ba0-b66a-d05699dfdebf","Type":"ContainerStarted","Data":"b8b6e60708e77d2b9b206ff6bd0d9ed5cfc39f7b0d7836562c1b2259d0ccc214"} Jul 28 16:02:34.509477 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.509467 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" event={"ID":"5d397052-5e4d-4ba0-b66a-d05699dfdebf","Type":"ContainerStarted","Data":"fc9d75b05df8474541fd3397fd4d4be4519468b4bf92b7cd9bd06ebe2e5e9180"} Jul 28 16:02:34.509558 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.509494 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" event={"ID":"5d397052-5e4d-4ba0-b66a-d05699dfdebf","Type":"ContainerStarted","Data":"07d5a316483eb571a6e49ad624ef3dc885137eaf50e62f09246c78ac5bcb793f"} Jul 28 16:02:34.509558 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.509546 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" event={"ID":"5d397052-5e4d-4ba0-b66a-d05699dfdebf","Type":"ContainerStarted","Data":"e26519326dd277e7c3f0d6f2cc342fa94ed89843c02d981f83d33c8b30a99e8a"} Jul 28 16:02:34.515360 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.515327 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pdxnl" podStartSLOduration=4.200642281 podStartE2EDuration="22.515316906s" totalImagesPullingTime="18.314674625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:34.514758464 +0000 UTC m=+22.548925792" watchObservedRunningTime="2026-07-28 16:02:34.515316906 +0000 UTC m=+22.549484232" Jul 28 16:02:34.609455 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.609229 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8xx66" podStartSLOduration=4.264700933 podStartE2EDuration="22.609212758s" totalImagesPullingTime="18.344511825s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:34.608808161 +0000 UTC m=+22.642975486" watchObservedRunningTime="2026-07-28 16:02:34.609212758 +0000 UTC m=+22.643380088" Jul 28 16:02:34.651877 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.651839 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-88478" podStartSLOduration=4.279039499 podStartE2EDuration="22.651827681s" totalImagesPullingTime="18.372788182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:34.651340874 +0000 UTC m=+22.685508206" watchObservedRunningTime="2026-07-28 16:02:34.651827681 +0000 UTC m=+22.685995006" Jul 28 16:02:34.692173 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.692085 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9nkv2" podStartSLOduration=4.390590847 podStartE2EDuration="22.6920708s" totalImagesPullingTime="18.301479953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:34.691700701 +0000 UTC m=+22.725868030" watchObservedRunningTime="2026-07-28 16:02:34.6920708 +0000 UTC m=+22.726238126" Jul 28 16:02:34.692289 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:34.692218 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jn6q8" podStartSLOduration=4.366814971 podStartE2EDuration="22.692212613s" totalImagesPullingTime="18.325397642s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:34.672114862 +0000 UTC m=+22.706282188" watchObservedRunningTime="2026-07-28 16:02:34.692212613 +0000 UTC m=+22.726379939" Jul 28 16:02:35.069013 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:35.068991 2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 16:02:35.370634 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:35.370517 2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T16:02:35.069008612Z","UUID":"a53bb1a9-1e81-47d9-991b-b006575c53bb","Handler":null,"Name":"","Endpoint":""} Jul 28 16:02:35.375225 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:35.375207 2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 16:02:35.375307 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:35.375241 2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 16:02:35.513217 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:35.513190 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal" event={"ID":"0d72e1a34529c7526e7d432be101f4be","Type":"ContainerStarted","Data":"3f7e2baa2ff61793727c2e11fcf85568c884280019750729eaf220574fc802d3"} Jul 28 16:02:35.514575 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:35.514554 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zx2dx" event={"ID":"2d065aea-c380-466a-bed3-80330b2c34b1","Type":"ContainerStarted","Data":"facc73284273bd9e1df1605acbc5156a0f84b576d7dbcb6ed4c21aab12a319b2"} Jul 28 16:02:35.516179 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:35.516157 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" event={"ID":"9a8a71ca-e316-42e4-87c8-d64a28caab24","Type":"ContainerStarted","Data":"1abfae00b69de6eaa1006b18155d3796aa3c7b300e49978a62f1667f51e27070"} Jul 28 16:02:35.536227 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:35.536189 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-10.ec2.internal" podStartSLOduration=22.536176394 podStartE2EDuration="22.536176394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:02:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:35.53604284 +0000 UTC m=+23.570210270" watchObservedRunningTime="2026-07-28 16:02:35.536176394 +0000 UTC m=+23.570343721" Jul 28 16:02:36.141305 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:36.141255 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:36.212844 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:36.212813 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:36.435738 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:36.435675 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:36.435878 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:36.435681 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:36.435878 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:36.435759 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:36.435878 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:36.435841 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:36.435998 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:36.435886 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:36.435998 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:36.435936 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:36.521072 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:36.521042 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" event={"ID":"5d397052-5e4d-4ba0-b66a-d05699dfdebf","Type":"ContainerStarted","Data":"2d050ad786fae2e919f93134af403c30005fbeb0f48ba6fa4d46d4e0ce7a71ff"} Jul 28 16:02:36.523000 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:36.522979 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" event={"ID":"9a8a71ca-e316-42e4-87c8-d64a28caab24","Type":"ContainerStarted","Data":"52b506aa867a70fa3b687ab1ec5f77bf2e962b58e61b21583b29c28398f8cda7"} Jul 28 16:02:36.556692 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:36.556654 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zx2dx" podStartSLOduration=6.239633727 podStartE2EDuration="24.556643697s" totalImagesPullingTime="18.31700997s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:35.558610943 +0000 UTC m=+23.592778269" watchObservedRunningTime="2026-07-28 16:02:36.556643697 +0000 UTC m=+24.590811024" Jul 28 16:02:36.610805 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:36.610787 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:37.027077 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:37.027055 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:37.122392 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:37.122365 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:37.796036 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:37.796007 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9nkv2" Jul 28 16:02:37.796646 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:37.796627 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9nkv2" Jul 28 16:02:37.820416 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:37.820358 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pk5kg" podStartSLOduration=5.183818821 podStartE2EDuration="25.820340384s" totalImagesPullingTime="20.636521563s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:02:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:36.557123712 +0000 UTC m=+24.591291040" watchObservedRunningTime="2026-07-28 16:02:37.820340384 +0000 UTC m=+25.854507709" Jul 28 16:02:38.436315 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:38.436084 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:38.436315 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:38.436084 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:38.436315 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:38.436205 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:38.436315 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:38.436264 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:38.436831 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:38.436322 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:38.436831 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:38.436423 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:38.531539 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:38.531505 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" event={"ID":"5d397052-5e4d-4ba0-b66a-d05699dfdebf","Type":"ContainerStarted","Data":"1a8011c5331128e5afa3dc4b03f40c2204e351e71152232bc8f64df74bfba72a"} Jul 28 16:02:38.531826 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:38.531804 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:38.531926 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:38.531882 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:38.547998 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:38.547948 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:38.578733 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:38.578688 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" podStartSLOduration=8.203273564 podStartE2EDuration="26.578670639s" totalImagesPullingTime="18.375397075s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:38.578667369 +0000 UTC m=+26.612834695" watchObservedRunningTime="2026-07-28 16:02:38.578670639 +0000 UTC m=+26.612837967" Jul 28 16:02:39.534521 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:39.534492 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:39.549571 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:39.549545 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:02:40.435829 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:40.435792 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:40.435829 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:40.435824 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:40.435995 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:40.435792 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:40.435995 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:40.435917 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:40.435995 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:40.435982 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:40.436107 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:40.436073 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:40.847074 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:40.846910 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hxxbk"] Jul 28 16:02:40.847455 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:40.847166 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:40.847455 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:40.847270 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:40.847725 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:40.847706 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nww4s"] Jul 28 16:02:40.847823 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:40.847791 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:40.847911 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:40.847891 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:40.848440 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:40.848397 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ngzg7"] Jul 28 16:02:40.848531 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:40.848510 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:40.848613 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:40.848590 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:41.540384 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:41.540350 2550 generic.go:356] "Generic (PLEG): container finished" podID="05a91de5-b17a-40da-96f0-1a0a25ca6c93" containerID="b37988f830df6d0055cc1d177d504b7540b1c1face8536b3d4d6e4bfa99e3d26" exitCode=0 Jul 28 16:02:41.540522 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:41.540441 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2ldwp" event={"ID":"05a91de5-b17a-40da-96f0-1a0a25ca6c93","Type":"ContainerDied","Data":"b37988f830df6d0055cc1d177d504b7540b1c1face8536b3d4d6e4bfa99e3d26"} Jul 28 16:02:42.436927 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:42.436904 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:42.437353 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:42.436997 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:42.437353 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:42.437015 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:42.437353 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:42.437058 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:42.437353 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:42.437082 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:42.437353 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:42.437203 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:43.005281 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:43.005247 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:43.542112 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:43.542078 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9nkv2" Jul 28 16:02:43.542568 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:43.542201 2550 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 28 16:02:43.544856 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:43.542791 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9nkv2" Jul 28 16:02:43.550805 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:43.550779 2550 generic.go:356] "Generic (PLEG): container finished" podID="05a91de5-b17a-40da-96f0-1a0a25ca6c93" containerID="3c2ea2ac5a18922447722c1b7622cbf9775f592ea0a44384d107cc989943ae83" exitCode=0 Jul 28 16:02:43.550903 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:43.550809 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2ldwp" event={"ID":"05a91de5-b17a-40da-96f0-1a0a25ca6c93","Type":"ContainerDied","Data":"3c2ea2ac5a18922447722c1b7622cbf9775f592ea0a44384d107cc989943ae83"} Jul 28 16:02:44.288746 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:44.288710 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:44.435860 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:44.435833 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:44.435996 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:44.435916 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxxbk" podUID="833bd299-67c9-4891-9d16-cdb458c0ffa1" Jul 28 16:02:44.435996 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:44.435923 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:44.435996 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:44.435986 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:44.436153 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:44.436076 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nww4s" podUID="676a8d84-5dd4-4207-91e6-a8256cdfa19e" Jul 28 16:02:44.436203 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:44.436170 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ngzg7" podUID="ddb31891-1896-4c34-97c8-6e45e42d349c" Jul 28 16:02:45.557101 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:45.557074 2550 generic.go:356] "Generic (PLEG): container finished" podID="05a91de5-b17a-40da-96f0-1a0a25ca6c93" containerID="55c0128ac20d16f069386c6f9bfb1eafe6f37fe13a1efa58f92821ad112b8ac1" exitCode=0 Jul 28 16:02:45.557468 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:45.557118 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2ldwp" event={"ID":"05a91de5-b17a-40da-96f0-1a0a25ca6c93","Type":"ContainerDied","Data":"55c0128ac20d16f069386c6f9bfb1eafe6f37fe13a1efa58f92821ad112b8ac1"} Jul 28 16:02:46.071464 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.071284 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs\") pod \"network-metrics-daemon-ngzg7\" (UID: \"ddb31891-1896-4c34-97c8-6e45e42d349c\") " pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:46.071585 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:46.071434 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:02:46.071585 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:46.071534 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs podName:ddb31891-1896-4c34-97c8-6e45e42d349c nodeName:}" failed. No retries permitted until 2026-07-28 16:03:18.071520811 +0000 UTC m=+66.105688115 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs") pod "network-metrics-daemon-ngzg7" (UID: "ddb31891-1896-4c34-97c8-6e45e42d349c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:02:46.171646 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.171627 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:46.171745 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:46.171704 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:02:46.171785 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:46.171745 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret podName:676a8d84-5dd4-4207-91e6-a8256cdfa19e nodeName:}" failed. No retries permitted until 2026-07-28 16:03:18.171733062 +0000 UTC m=+66.205900366 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret") pod "global-pull-secret-syncer-nww4s" (UID: "676a8d84-5dd4-4207-91e6-a8256cdfa19e") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:02:46.272906 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.272830 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvkqd\" (UniqueName: \"kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd\") pod \"network-check-target-hxxbk\" (UID: \"833bd299-67c9-4891-9d16-cdb458c0ffa1\") " pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:46.273013 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:46.272969 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:02:46.273013 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:46.272987 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:02:46.273013 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:46.272999 2550 projected.go:196] Error preparing data for projected volume kube-api-access-fvkqd for pod openshift-network-diagnostics/network-check-target-hxxbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:02:46.273129 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:46.273053 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd podName:833bd299-67c9-4891-9d16-cdb458c0ffa1 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:18.273037929 +0000 UTC m=+66.307205237 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fvkqd" (UniqueName: "kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd") pod "network-check-target-hxxbk" (UID: "833bd299-67c9-4891-9d16-cdb458c0ffa1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:02:46.273489 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.273470 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-10.ec2.internal" event="NodeReady" Jul 28 16:02:46.273619 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.273606 2550 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 16:02:46.333939 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.333897 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-28lcc"] Jul 28 16:02:46.357087 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.356642 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b5jtv"] Jul 28 16:02:46.357255 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.357238 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:46.363303 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.363280 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 16:02:46.363303 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.363298 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ssj24\"" Jul 28 16:02:46.363539 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.363281 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 16:02:46.379962 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.379944 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-28lcc"] Jul 28 16:02:46.380060 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.379982 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b5jtv"] Jul 28 16:02:46.380104 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.380066 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b5jtv" Jul 28 16:02:46.382870 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.382850 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 16:02:46.382870 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.382862 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 16:02:46.383155 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.383139 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 16:02:46.383301 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.383279 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bjngx\"" Jul 28 16:02:46.435848 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.435829 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:02:46.435848 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.435846 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:02:46.436018 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.435857 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:02:46.440916 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.440898 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 16:02:46.440916 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.440927 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bvgkt\"" Jul 28 16:02:46.441100 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.440990 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 16:02:46.441100 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.441012 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f7pxp\"" Jul 28 16:02:46.441248 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.441188 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 16:02:46.441350 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.441321 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 16:02:46.474990 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.474967 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ecf78f-a024-4986-888d-67b82853a16b-tmp-dir\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:46.475090 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.475001 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:46.475090 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.475037 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d4ecf78f-a024-4986-888d-67b82853a16b-config-volume\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:46.475190 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.475115 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9qds\" (UniqueName: \"kubernetes.io/projected/d4ecf78f-a024-4986-888d-67b82853a16b-kube-api-access-p9qds\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:46.475240 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.475196 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgkzm\" (UniqueName: \"kubernetes.io/projected/dd3769ac-7034-4a90-9100-03479a58e78a-kube-api-access-zgkzm\") pod \"ingress-canary-b5jtv\" (UID: \"dd3769ac-7034-4a90-9100-03479a58e78a\") " pod="openshift-ingress-canary/ingress-canary-b5jtv" Jul 28 16:02:46.475240 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.475217 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd3769ac-7034-4a90-9100-03479a58e78a-cert\") pod \"ingress-canary-b5jtv\" (UID: \"dd3769ac-7034-4a90-9100-03479a58e78a\") " pod="openshift-ingress-canary/ingress-canary-b5jtv" Jul 28 16:02:46.558363 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.558278 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:46.576497 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.576477 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgkzm\" (UniqueName: \"kubernetes.io/projected/dd3769ac-7034-4a90-9100-03479a58e78a-kube-api-access-zgkzm\") pod \"ingress-canary-b5jtv\" (UID: \"dd3769ac-7034-4a90-9100-03479a58e78a\") " pod="openshift-ingress-canary/ingress-canary-b5jtv" Jul 28 16:02:46.576608 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.576508 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd3769ac-7034-4a90-9100-03479a58e78a-cert\") pod \"ingress-canary-b5jtv\" (UID: \"dd3769ac-7034-4a90-9100-03479a58e78a\") " pod="openshift-ingress-canary/ingress-canary-b5jtv" Jul 28 16:02:46.576789 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.576771 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ecf78f-a024-4986-888d-67b82853a16b-tmp-dir\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:46.576860 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.576805 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:46.576912 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.576858 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d4ecf78f-a024-4986-888d-67b82853a16b-config-volume\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:46.576957 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:46.576929 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 16:02:46.577002 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:46.576994 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls podName:d4ecf78f-a024-4986-888d-67b82853a16b nodeName:}" failed. No retries permitted until 2026-07-28 16:02:47.07697581 +0000 UTC m=+35.111143119 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls") pod "dns-default-28lcc" (UID: "d4ecf78f-a024-4986-888d-67b82853a16b") : secret "dns-default-metrics-tls" not found Jul 28 16:02:46.577141 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.577117 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ecf78f-a024-4986-888d-67b82853a16b-tmp-dir\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:46.577246 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.577190 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9qds\" (UniqueName: \"kubernetes.io/projected/d4ecf78f-a024-4986-888d-67b82853a16b-kube-api-access-p9qds\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:46.577436 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.577397 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d4ecf78f-a024-4986-888d-67b82853a16b-config-volume\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:46.581071 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.581054 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd3769ac-7034-4a90-9100-03479a58e78a-cert\") pod \"ingress-canary-b5jtv\" (UID: \"dd3769ac-7034-4a90-9100-03479a58e78a\") " pod="openshift-ingress-canary/ingress-canary-b5jtv" Jul 28 16:02:46.597469 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.597452 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgkzm\" (UniqueName: \"kubernetes.io/projected/dd3769ac-7034-4a90-9100-03479a58e78a-kube-api-access-zgkzm\") pod \"ingress-canary-b5jtv\" (UID: \"dd3769ac-7034-4a90-9100-03479a58e78a\") " pod="openshift-ingress-canary/ingress-canary-b5jtv" Jul 28 16:02:46.598671 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.598653 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9qds\" (UniqueName: \"kubernetes.io/projected/d4ecf78f-a024-4986-888d-67b82853a16b-kube-api-access-p9qds\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:46.657302 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.657276 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:46.690491 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.690469 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b5jtv" Jul 28 16:02:46.900354 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:46.900323 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b5jtv"] Jul 28 16:02:46.903779 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:02:46.903747 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd3769ac_7034_4a90_9100_03479a58e78a.slice/crio-8d7a1fbd0f2e407c38d07ce69b968e756e3a93f97ddf8537059d83d2c91cdbd9 WatchSource:0}: Error finding container 8d7a1fbd0f2e407c38d07ce69b968e756e3a93f97ddf8537059d83d2c91cdbd9: Status 404 returned error can't find the container with id 8d7a1fbd0f2e407c38d07ce69b968e756e3a93f97ddf8537059d83d2c91cdbd9 Jul 28 16:02:47.081250 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:47.081223 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:47.081457 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:47.081380 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 16:02:47.081457 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:47.081455 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls podName:d4ecf78f-a024-4986-888d-67b82853a16b nodeName:}" failed. No retries permitted until 2026-07-28 16:02:48.08143829 +0000 UTC m=+36.115605598 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls") pod "dns-default-28lcc" (UID: "d4ecf78f-a024-4986-888d-67b82853a16b") : secret "dns-default-metrics-tls" not found Jul 28 16:02:47.563897 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:47.563842 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b5jtv" event={"ID":"dd3769ac-7034-4a90-9100-03479a58e78a","Type":"ContainerStarted","Data":"8d7a1fbd0f2e407c38d07ce69b968e756e3a93f97ddf8537059d83d2c91cdbd9"} Jul 28 16:02:48.087763 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:48.087726 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:48.087967 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:48.087886 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 16:02:48.087967 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:48.087959 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls podName:d4ecf78f-a024-4986-888d-67b82853a16b nodeName:}" failed. No retries permitted until 2026-07-28 16:02:50.087940248 +0000 UTC m=+38.122107554 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls") pod "dns-default-28lcc" (UID: "d4ecf78f-a024-4986-888d-67b82853a16b") : secret "dns-default-metrics-tls" not found Jul 28 16:02:50.102029 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:50.101824 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:50.102487 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:50.101959 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 16:02:50.102487 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:50.102146 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls podName:d4ecf78f-a024-4986-888d-67b82853a16b nodeName:}" failed. No retries permitted until 2026-07-28 16:02:54.10213035 +0000 UTC m=+42.136297654 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls") pod "dns-default-28lcc" (UID: "d4ecf78f-a024-4986-888d-67b82853a16b") : secret "dns-default-metrics-tls" not found Jul 28 16:02:51.576798 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:51.576766 2550 generic.go:356] "Generic (PLEG): container finished" podID="05a91de5-b17a-40da-96f0-1a0a25ca6c93" containerID="9db3380fa494ad754d166a3bd1c4635e23c0d547de7592849bb36a86f1ef157a" exitCode=0 Jul 28 16:02:51.577196 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:51.576853 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2ldwp" event={"ID":"05a91de5-b17a-40da-96f0-1a0a25ca6c93","Type":"ContainerDied","Data":"9db3380fa494ad754d166a3bd1c4635e23c0d547de7592849bb36a86f1ef157a"} Jul 28 16:02:51.578511 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:51.578484 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b5jtv" event={"ID":"dd3769ac-7034-4a90-9100-03479a58e78a","Type":"ContainerStarted","Data":"e14ac86873bef64c3e09bbbc53fcd7d73401e144b703ded147bde7169c92174c"} Jul 28 16:02:51.615598 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:51.615556 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b5jtv" podStartSLOduration=1.223204385 podStartE2EDuration="5.615538628s" totalImagesPullingTime="4.392334243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:02:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:51.615137982 +0000 UTC m=+39.649305309" watchObservedRunningTime="2026-07-28 16:02:51.615538628 +0000 UTC m=+39.649705954" Jul 28 16:02:51.873323 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:51.873261 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:52.583631 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:52.583585 2550 generic.go:356] "Generic (PLEG): container finished" podID="05a91de5-b17a-40da-96f0-1a0a25ca6c93" containerID="7e7417a134b33f7a826d3b00a748ab91de7a5f0347d21a00327a35f352e98f26" exitCode=0 Jul 28 16:02:52.584019 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:52.583678 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2ldwp" event={"ID":"05a91de5-b17a-40da-96f0-1a0a25ca6c93","Type":"ContainerDied","Data":"7e7417a134b33f7a826d3b00a748ab91de7a5f0347d21a00327a35f352e98f26"} Jul 28 16:02:53.589908 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:53.589862 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2ldwp" event={"ID":"05a91de5-b17a-40da-96f0-1a0a25ca6c93","Type":"ContainerStarted","Data":"67afe1308bbe286eec01b640610bd2f5c9e4d48d15f53013d4e51fdd35c737b1"} Jul 28 16:02:53.619425 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:53.619362 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2ldwp" podStartSLOduration=9.022203097 podStartE2EDuration="41.61934452s" totalImagesPullingTime="32.597141423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:02:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:02:53.618110803 +0000 UTC m=+41.652278129" watchObservedRunningTime="2026-07-28 16:02:53.61934452 +0000 UTC m=+41.653511845" Jul 28 16:02:53.762856 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:53.762824 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:53.830314 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:53.830286 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:54.128804 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:54.128774 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:02:54.128931 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:54.128898 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 16:02:54.129003 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:02:54.128968 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls podName:d4ecf78f-a024-4986-888d-67b82853a16b nodeName:}" failed. No retries permitted until 2026-07-28 16:03:02.128948538 +0000 UTC m=+50.163115856 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls") pod "dns-default-28lcc" (UID: "d4ecf78f-a024-4986-888d-67b82853a16b") : secret "dns-default-metrics-tls" not found Jul 28 16:02:54.396907 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:54.396844 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:54.500468 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:54.500442 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:58.246021 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:58.245979 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:02:58.313231 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:02:58.313198 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:01.230524 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:01.230478 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:02.178112 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:02.178075 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:03:02.178355 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:02.178190 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 16:03:02.178355 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:02.178252 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls podName:d4ecf78f-a024-4986-888d-67b82853a16b nodeName:}" failed. No retries permitted until 2026-07-28 16:03:18.178234184 +0000 UTC m=+66.212401494 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls") pod "dns-default-28lcc" (UID: "d4ecf78f-a024-4986-888d-67b82853a16b") : secret "dns-default-metrics-tls" not found Jul 28 16:03:08.565794 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:08.565756 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:11.552312 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:11.552267 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4xnwq" Jul 28 16:03:11.714645 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:11.714614 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:11.783023 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:11.782993 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:11.841467 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:11.841398 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:13.781561 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:13.781524 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:17.339984 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:17.339944 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:17.431034 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:17.431001 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:18.082345 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.082312 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs\") pod \"network-metrics-daemon-ngzg7\" (UID: \"ddb31891-1896-4c34-97c8-6e45e42d349c\") " pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:03:18.084901 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.084881 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 16:03:18.093534 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:18.093513 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 28 16:03:18.093604 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:18.093589 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs podName:ddb31891-1896-4c34-97c8-6e45e42d349c nodeName:}" failed. No retries permitted until 2026-07-28 16:04:22.093569274 +0000 UTC m=+130.127736595 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs") pod "network-metrics-daemon-ngzg7" (UID: "ddb31891-1896-4c34-97c8-6e45e42d349c") : secret "metrics-daemon-secret" not found Jul 28 16:03:18.183233 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.183196 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:03:18.183343 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.183248 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:03:18.183402 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:18.183343 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 16:03:18.183471 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:18.183422 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls podName:d4ecf78f-a024-4986-888d-67b82853a16b nodeName:}" failed. No retries permitted until 2026-07-28 16:03:50.183391874 +0000 UTC m=+98.217559177 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls") pod "dns-default-28lcc" (UID: "d4ecf78f-a024-4986-888d-67b82853a16b") : secret "dns-default-metrics-tls" not found Jul 28 16:03:18.185399 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.185381 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 16:03:18.197341 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.197316 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/676a8d84-5dd4-4207-91e6-a8256cdfa19e-original-pull-secret\") pod \"global-pull-secret-syncer-nww4s\" (UID: \"676a8d84-5dd4-4207-91e6-a8256cdfa19e\") " pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:03:18.260506 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.260483 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nww4s" Jul 28 16:03:18.284026 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.284002 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvkqd\" (UniqueName: \"kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd\") pod \"network-check-target-hxxbk\" (UID: \"833bd299-67c9-4891-9d16-cdb458c0ffa1\") " pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:03:18.286608 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.286590 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 16:03:18.296519 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.296497 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 16:03:18.309065 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.309024 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvkqd\" (UniqueName: \"kubernetes.io/projected/833bd299-67c9-4891-9d16-cdb458c0ffa1-kube-api-access-fvkqd\") pod \"network-check-target-hxxbk\" (UID: \"833bd299-67c9-4891-9d16-cdb458c0ffa1\") " pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:03:18.382897 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.382873 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nww4s"] Jul 28 16:03:18.562987 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.562962 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bvgkt\"" Jul 28 16:03:18.571461 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.571439 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:03:18.648537 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.648502 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nww4s" event={"ID":"676a8d84-5dd4-4207-91e6-a8256cdfa19e","Type":"ContainerStarted","Data":"b887c17d914f27af3a5866a76ee2231ce243a1db06eb693e5d3df750e940d86e"} Jul 28 16:03:18.693169 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:18.693144 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hxxbk"] Jul 28 16:03:18.696148 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:03:18.696124 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod833bd299_67c9_4891_9d16_cdb458c0ffa1.slice/crio-b05cc8b0e3952e5efd487027f4fcbdddb0b40911d545c5c9bfa2c53a5b58d8b2 WatchSource:0}: Error finding container b05cc8b0e3952e5efd487027f4fcbdddb0b40911d545c5c9bfa2c53a5b58d8b2: Status 404 returned error can't find the container with id b05cc8b0e3952e5efd487027f4fcbdddb0b40911d545c5c9bfa2c53a5b58d8b2 Jul 28 16:03:19.652976 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:19.652903 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hxxbk" event={"ID":"833bd299-67c9-4891-9d16-cdb458c0ffa1","Type":"ContainerStarted","Data":"b05cc8b0e3952e5efd487027f4fcbdddb0b40911d545c5c9bfa2c53a5b58d8b2"} Jul 28 16:03:22.152452 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.152418 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-lxlwg"] Jul 28 16:03:22.155417 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.155389 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.157728 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.157694 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-jh2t9\"" Jul 28 16:03:22.157869 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.157694 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 28 16:03:22.157869 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.157792 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 16:03:22.157869 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.157697 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 28 16:03:22.158548 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.158515 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 16:03:22.160476 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.160453 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7f859bc589-qkbls"] Jul 28 16:03:22.162947 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.162929 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 28 16:03:22.164198 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.163808 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.164317 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.164215 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-lxlwg"] Jul 28 16:03:22.166191 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.166147 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 28 16:03:22.166317 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.166199 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 28 16:03:22.166386 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.166375 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6p68w\"" Jul 28 16:03:22.166611 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.166480 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 28 16:03:22.166611 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.166593 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 28 16:03:22.167457 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.167436 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 28 16:03:22.167845 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.167819 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 28 16:03:22.176426 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.176384 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f859bc589-qkbls"] Jul 28 16:03:22.206803 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.206774 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-default-certificate\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.206928 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.206815 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwjml\" (UniqueName: \"kubernetes.io/projected/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-kube-api-access-fwjml\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.206928 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.206849 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.206928 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.206872 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f454402-1f88-4231-a0f5-5bc2f58d0a01-tmp\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.206928 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.206898 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-stats-auth\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.206928 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.206921 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f454402-1f88-4231-a0f5-5bc2f58d0a01-service-ca-bundle\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.207185 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.206982 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.207185 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.207015 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8f454402-1f88-4231-a0f5-5bc2f58d0a01-snapshots\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.207185 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.207038 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f454402-1f88-4231-a0f5-5bc2f58d0a01-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.207185 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.207062 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f454402-1f88-4231-a0f5-5bc2f58d0a01-serving-cert\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.207185 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.207083 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46cxx\" (UniqueName: \"kubernetes.io/projected/8f454402-1f88-4231-a0f5-5bc2f58d0a01-kube-api-access-46cxx\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.307572 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.307544 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.307719 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.307579 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f454402-1f88-4231-a0f5-5bc2f58d0a01-tmp\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.307719 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.307604 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-stats-auth\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.307719 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.307625 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f454402-1f88-4231-a0f5-5bc2f58d0a01-service-ca-bundle\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.307719 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.307681 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.307719 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:22.307699 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 16:03:22.307719 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.307714 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8f454402-1f88-4231-a0f5-5bc2f58d0a01-snapshots\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.307982 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:22.307768 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs podName:6c1c9a74-abad-430a-ad3f-fe0d70833ca6 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:22.807746001 +0000 UTC m=+70.841913323 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs") pod "router-default-7f859bc589-qkbls" (UID: "6c1c9a74-abad-430a-ad3f-fe0d70833ca6") : secret "router-metrics-certs-default" not found Jul 28 16:03:22.307982 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:22.307814 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle podName:6c1c9a74-abad-430a-ad3f-fe0d70833ca6 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:22.80780076 +0000 UTC m=+70.841968069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle") pod "router-default-7f859bc589-qkbls" (UID: "6c1c9a74-abad-430a-ad3f-fe0d70833ca6") : configmap references non-existent config key: service-ca.crt Jul 28 16:03:22.307982 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.307847 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f454402-1f88-4231-a0f5-5bc2f58d0a01-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.307982 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.307878 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f454402-1f88-4231-a0f5-5bc2f58d0a01-serving-cert\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.307982 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.307901 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46cxx\" (UniqueName: \"kubernetes.io/projected/8f454402-1f88-4231-a0f5-5bc2f58d0a01-kube-api-access-46cxx\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.307982 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.307957 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-default-certificate\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.307982 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.307982 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwjml\" (UniqueName: \"kubernetes.io/projected/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-kube-api-access-fwjml\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.308326 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.308075 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f454402-1f88-4231-a0f5-5bc2f58d0a01-tmp\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.308451 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.308393 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f454402-1f88-4231-a0f5-5bc2f58d0a01-service-ca-bundle\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.308594 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.308569 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8f454402-1f88-4231-a0f5-5bc2f58d0a01-snapshots\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.308953 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.308903 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f454402-1f88-4231-a0f5-5bc2f58d0a01-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.310381 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.310354 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-stats-auth\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.310484 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.310390 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f454402-1f88-4231-a0f5-5bc2f58d0a01-serving-cert\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.310895 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.310875 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-default-certificate\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.316846 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.316824 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwjml\" (UniqueName: \"kubernetes.io/projected/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-kube-api-access-fwjml\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.316950 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.316852 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46cxx\" (UniqueName: \"kubernetes.io/projected/8f454402-1f88-4231-a0f5-5bc2f58d0a01-kube-api-access-46cxx\") pod \"insights-operator-5885fdb759-lxlwg\" (UID: \"8f454402-1f88-4231-a0f5-5bc2f58d0a01\") " pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.329376 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.329354 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:22.388633 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.388602 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:22.467489 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.467431 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-lxlwg" Jul 28 16:03:22.812291 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.812205 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.812465 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:22.812292 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:22.812465 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:22.812386 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle podName:6c1c9a74-abad-430a-ad3f-fe0d70833ca6 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:23.812365674 +0000 UTC m=+71.846532981 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle") pod "router-default-7f859bc589-qkbls" (UID: "6c1c9a74-abad-430a-ad3f-fe0d70833ca6") : configmap references non-existent config key: service-ca.crt Jul 28 16:03:22.812465 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:22.812398 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 16:03:22.812465 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:22.812464 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs podName:6c1c9a74-abad-430a-ad3f-fe0d70833ca6 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:23.812450757 +0000 UTC m=+71.846618080 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs") pod "router-default-7f859bc589-qkbls" (UID: "6c1c9a74-abad-430a-ad3f-fe0d70833ca6") : secret "router-metrics-certs-default" not found Jul 28 16:03:23.146972 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:23.146945 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-lxlwg"] Jul 28 16:03:23.150066 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:03:23.150038 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f454402_1f88_4231_a0f5_5bc2f58d0a01.slice/crio-09d581d3563000f83c5695fcaf4dfd8f1d8950f5168878bfbc554992c4703470 WatchSource:0}: Error finding container 09d581d3563000f83c5695fcaf4dfd8f1d8950f5168878bfbc554992c4703470: Status 404 returned error can't find the container with id 09d581d3563000f83c5695fcaf4dfd8f1d8950f5168878bfbc554992c4703470 Jul 28 16:03:23.664788 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:23.664750 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nww4s" event={"ID":"676a8d84-5dd4-4207-91e6-a8256cdfa19e","Type":"ContainerStarted","Data":"4efc404016955227ac76f6800db0acc81f043028708698e594ce3c854d715365"} Jul 28 16:03:23.666325 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:23.666298 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hxxbk" event={"ID":"833bd299-67c9-4891-9d16-cdb458c0ffa1","Type":"ContainerStarted","Data":"0f194b25db8d4cddb6083f2bc216ee319e7f86ddbf0f57a374d51c9615386f08"} Jul 28 16:03:23.666457 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:23.666443 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:03:23.667480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:23.667459 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-lxlwg" event={"ID":"8f454402-1f88-4231-a0f5-5bc2f58d0a01","Type":"ContainerStarted","Data":"09d581d3563000f83c5695fcaf4dfd8f1d8950f5168878bfbc554992c4703470"} Jul 28 16:03:23.682720 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:23.682679 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nww4s" podStartSLOduration=67.039288018 podStartE2EDuration="1m11.682666116s" totalImagesPullingTime="4.643378098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:03:23.682655811 +0000 UTC m=+71.716823136" watchObservedRunningTime="2026-07-28 16:03:23.682666116 +0000 UTC m=+71.716833432" Jul 28 16:03:23.700522 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:23.700480 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hxxbk" podStartSLOduration=67.371026745 podStartE2EDuration="1m11.700467272s" totalImagesPullingTime="4.329440527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:03:23.698968776 +0000 UTC m=+71.733136101" watchObservedRunningTime="2026-07-28 16:03:23.700467272 +0000 UTC m=+71.734634589" Jul 28 16:03:23.819389 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:23.819361 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:23.819510 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:23.819428 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:23.819510 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:23.819492 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 16:03:23.819592 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:23.819526 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle podName:6c1c9a74-abad-430a-ad3f-fe0d70833ca6 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:25.819514104 +0000 UTC m=+73.853681408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle") pod "router-default-7f859bc589-qkbls" (UID: "6c1c9a74-abad-430a-ad3f-fe0d70833ca6") : configmap references non-existent config key: service-ca.crt Jul 28 16:03:23.819592 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:23.819539 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs podName:6c1c9a74-abad-430a-ad3f-fe0d70833ca6 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:25.819533849 +0000 UTC m=+73.853701153 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs") pod "router-default-7f859bc589-qkbls" (UID: "6c1c9a74-abad-430a-ad3f-fe0d70833ca6") : secret "router-metrics-certs-default" not found Jul 28 16:03:25.673376 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:25.673339 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-lxlwg" event={"ID":"8f454402-1f88-4231-a0f5-5bc2f58d0a01","Type":"ContainerStarted","Data":"8d1555d8359edf3201995e8d5770aa1f0cd9a3f888cc813a610aadee19d13118"} Jul 28 16:03:25.693871 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:25.693828 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-lxlwg" podStartSLOduration=1.308318153 podStartE2EDuration="3.6938104s" totalImagesPullingTime="2.385492247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:03:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:03:25.6929652 +0000 UTC m=+73.727132527" watchObservedRunningTime="2026-07-28 16:03:25.6938104 +0000 UTC m=+73.727977725" Jul 28 16:03:25.833247 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:25.833226 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:25.833352 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:25.833271 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:25.833391 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:25.833345 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 16:03:25.833391 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:25.833365 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle podName:6c1c9a74-abad-430a-ad3f-fe0d70833ca6 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:29.833353336 +0000 UTC m=+77.867520640 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle") pod "router-default-7f859bc589-qkbls" (UID: "6c1c9a74-abad-430a-ad3f-fe0d70833ca6") : configmap references non-existent config key: service-ca.crt Jul 28 16:03:25.833481 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:25.833391 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs podName:6c1c9a74-abad-430a-ad3f-fe0d70833ca6 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:29.833378988 +0000 UTC m=+77.867546292 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs") pod "router-default-7f859bc589-qkbls" (UID: "6c1c9a74-abad-430a-ad3f-fe0d70833ca6") : secret "router-metrics-certs-default" not found Jul 28 16:03:27.535361 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.535329 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8r9q6"] Jul 28 16:03:27.537310 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.537286 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:27.540177 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.540131 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 16:03:27.540330 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.540314 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nbttq\"" Jul 28 16:03:27.540434 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.540318 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 16:03:27.543597 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.543576 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6084bd94-c631-4f0f-a5a5-869566f1b430-crio-socket\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:27.543735 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.543615 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6nzf\" (UniqueName: \"kubernetes.io/projected/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-api-access-h6nzf\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:27.543735 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.543662 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:27.543875 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.543809 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:27.544183 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.543914 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6084bd94-c631-4f0f-a5a5-869566f1b430-data-volume\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:27.549602 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.549579 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8r9q6"] Jul 28 16:03:27.563041 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.563022 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:27.620948 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.620927 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:27.644902 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.644877 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6084bd94-c631-4f0f-a5a5-869566f1b430-data-volume\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:27.644986 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.644911 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6084bd94-c631-4f0f-a5a5-869566f1b430-crio-socket\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:27.644986 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.644935 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6nzf\" (UniqueName: \"kubernetes.io/projected/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-api-access-h6nzf\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:27.645092 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.644996 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6084bd94-c631-4f0f-a5a5-869566f1b430-crio-socket\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:27.645145 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.645109 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:27.645198 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.645151 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:27.645251 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:27.645226 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 16:03:27.645251 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.645235 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6084bd94-c631-4f0f-a5a5-869566f1b430-data-volume\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:27.645330 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:27.645254 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 16:03:27.645330 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:27.645283 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls podName:6084bd94-c631-4f0f-a5a5-869566f1b430 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:28.14526711 +0000 UTC m=+76.179434430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8r9q6" (UID: "6084bd94-c631-4f0f-a5a5-869566f1b430") : secret "insights-runtime-extractor-tls" not found Jul 28 16:03:27.645330 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:27.645297 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm podName:6084bd94-c631-4f0f-a5a5-869566f1b430 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:28.145291303 +0000 UTC m=+76.179458607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8r9q6" (UID: "6084bd94-c631-4f0f-a5a5-869566f1b430") : configmap "kube-rbac-proxy" not found Jul 28 16:03:27.655545 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:27.655527 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6nzf\" (UniqueName: \"kubernetes.io/projected/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-api-access-h6nzf\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:28.147693 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:28.147665 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:28.147857 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:28.147723 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:28.147857 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:28.147812 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 16:03:28.147857 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:28.147829 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 16:03:28.147991 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:28.147875 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls podName:6084bd94-c631-4f0f-a5a5-869566f1b430 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:29.147860551 +0000 UTC m=+77.182027855 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8r9q6" (UID: "6084bd94-c631-4f0f-a5a5-869566f1b430") : secret "insights-runtime-extractor-tls" not found Jul 28 16:03:28.147991 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:28.147900 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm podName:6084bd94-c631-4f0f-a5a5-869566f1b430 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:29.147881431 +0000 UTC m=+77.182048735 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8r9q6" (UID: "6084bd94-c631-4f0f-a5a5-869566f1b430") : configmap "kube-rbac-proxy" not found Jul 28 16:03:29.156014 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:29.155975 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:29.156373 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:29.156078 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:29.156373 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:29.156112 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 16:03:29.156373 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:29.156211 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm podName:6084bd94-c631-4f0f-a5a5-869566f1b430 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:31.156193775 +0000 UTC m=+79.190361082 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8r9q6" (UID: "6084bd94-c631-4f0f-a5a5-869566f1b430") : configmap "kube-rbac-proxy" not found Jul 28 16:03:29.156373 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:29.156217 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 16:03:29.156373 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:29.156280 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls podName:6084bd94-c631-4f0f-a5a5-869566f1b430 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:31.156263006 +0000 UTC m=+79.190430330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8r9q6" (UID: "6084bd94-c631-4f0f-a5a5-869566f1b430") : secret "insights-runtime-extractor-tls" not found Jul 28 16:03:29.712714 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:29.712688 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jn6q8_afa6eae0-4d40-4f40-9358-69feb4831e42/dns-node-resolver/0.log" Jul 28 16:03:29.861611 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:29.861579 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:29.861717 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:29.861640 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:29.861760 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:29.861721 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 16:03:29.861798 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:29.861761 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle podName:6c1c9a74-abad-430a-ad3f-fe0d70833ca6 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:37.861747932 +0000 UTC m=+85.895915236 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle") pod "router-default-7f859bc589-qkbls" (UID: "6c1c9a74-abad-430a-ad3f-fe0d70833ca6") : configmap references non-existent config key: service-ca.crt Jul 28 16:03:29.861798 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:29.861777 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs podName:6c1c9a74-abad-430a-ad3f-fe0d70833ca6 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:37.861770791 +0000 UTC m=+85.895938095 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs") pod "router-default-7f859bc589-qkbls" (UID: "6c1c9a74-abad-430a-ad3f-fe0d70833ca6") : secret "router-metrics-certs-default" not found Jul 28 16:03:30.312207 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:30.312182 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pdxnl_088caeed-2739-4bd0-836b-3a7b0e456194/node-ca/0.log" Jul 28 16:03:30.912494 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:30.912468 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b5jtv_dd3769ac-7034-4a90-9100-03479a58e78a/serve-healthcheck-canary/0.log" Jul 28 16:03:31.171384 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:31.171324 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:31.171384 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:31.171369 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:31.171555 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:31.171481 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 16:03:31.171555 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:31.171513 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 16:03:31.171555 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:31.171541 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls podName:6084bd94-c631-4f0f-a5a5-869566f1b430 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:35.171527225 +0000 UTC m=+83.205694532 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8r9q6" (UID: "6084bd94-c631-4f0f-a5a5-869566f1b430") : secret "insights-runtime-extractor-tls" not found Jul 28 16:03:31.171654 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:31.171565 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm podName:6084bd94-c631-4f0f-a5a5-869566f1b430 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:35.171552475 +0000 UTC m=+83.205719779 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8r9q6" (UID: "6084bd94-c631-4f0f-a5a5-869566f1b430") : configmap "kube-rbac-proxy" not found Jul 28 16:03:32.139468 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:32.139436 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8"] Jul 28 16:03:32.141430 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:32.141391 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" Jul 28 16:03:32.143812 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:32.143788 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 28 16:03:32.143920 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:32.143811 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 28 16:03:32.143920 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:32.143814 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 28 16:03:32.144452 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:32.144434 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jszf5\"" Jul 28 16:03:32.151813 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:32.151782 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8"] Jul 28 16:03:32.177393 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:32.177370 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-g5sp8\" (UID: \"3345d4cb-2854-4bf9-a611-7f376f949dc0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" Jul 28 16:03:32.177522 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:32.177397 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxsx6\" (UniqueName: \"kubernetes.io/projected/3345d4cb-2854-4bf9-a611-7f376f949dc0-kube-api-access-dxsx6\") pod \"cluster-samples-operator-6748489656-g5sp8\" (UID: \"3345d4cb-2854-4bf9-a611-7f376f949dc0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" Jul 28 16:03:32.278036 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:32.278006 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-g5sp8\" (UID: \"3345d4cb-2854-4bf9-a611-7f376f949dc0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" Jul 28 16:03:32.278036 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:32.278036 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxsx6\" (UniqueName: \"kubernetes.io/projected/3345d4cb-2854-4bf9-a611-7f376f949dc0-kube-api-access-dxsx6\") pod \"cluster-samples-operator-6748489656-g5sp8\" (UID: \"3345d4cb-2854-4bf9-a611-7f376f949dc0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" Jul 28 16:03:32.278178 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:32.278155 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 16:03:32.278251 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:32.278238 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls podName:3345d4cb-2854-4bf9-a611-7f376f949dc0 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:32.778222988 +0000 UTC m=+80.812390292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls") pod "cluster-samples-operator-6748489656-g5sp8" (UID: "3345d4cb-2854-4bf9-a611-7f376f949dc0") : secret "samples-operator-tls" not found Jul 28 16:03:32.287144 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:32.287115 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxsx6\" (UniqueName: \"kubernetes.io/projected/3345d4cb-2854-4bf9-a611-7f376f949dc0-kube-api-access-dxsx6\") pod \"cluster-samples-operator-6748489656-g5sp8\" (UID: \"3345d4cb-2854-4bf9-a611-7f376f949dc0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" Jul 28 16:03:32.782627 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:32.782593 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-g5sp8\" (UID: \"3345d4cb-2854-4bf9-a611-7f376f949dc0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" Jul 28 16:03:32.782764 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:32.782707 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 16:03:32.782764 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:32.782760 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls podName:3345d4cb-2854-4bf9-a611-7f376f949dc0 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:33.782746316 +0000 UTC m=+81.816913620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls") pod "cluster-samples-operator-6748489656-g5sp8" (UID: "3345d4cb-2854-4bf9-a611-7f376f949dc0") : secret "samples-operator-tls" not found Jul 28 16:03:33.789623 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:33.789589 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-g5sp8\" (UID: \"3345d4cb-2854-4bf9-a611-7f376f949dc0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" Jul 28 16:03:33.789972 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:33.789711 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 16:03:33.789972 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:33.789766 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls podName:3345d4cb-2854-4bf9-a611-7f376f949dc0 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:35.789750491 +0000 UTC m=+83.823917795 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls") pod "cluster-samples-operator-6748489656-g5sp8" (UID: "3345d4cb-2854-4bf9-a611-7f376f949dc0") : secret "samples-operator-tls" not found Jul 28 16:03:34.271990 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:34.271883 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:34.373014 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:34.372989 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:35.200450 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:35.200421 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:35.200831 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:35.200502 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:35.200831 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:35.200563 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 16:03:35.200831 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:35.200592 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 16:03:35.200831 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:35.200640 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls podName:6084bd94-c631-4f0f-a5a5-869566f1b430 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:43.200627558 +0000 UTC m=+91.234794861 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8r9q6" (UID: "6084bd94-c631-4f0f-a5a5-869566f1b430") : secret "insights-runtime-extractor-tls" not found Jul 28 16:03:35.200831 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:35.200653 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm podName:6084bd94-c631-4f0f-a5a5-869566f1b430 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:43.200646522 +0000 UTC m=+91.234813826 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8r9q6" (UID: "6084bd94-c631-4f0f-a5a5-869566f1b430") : configmap "kube-rbac-proxy" not found Jul 28 16:03:35.805718 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:35.805686 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-g5sp8\" (UID: \"3345d4cb-2854-4bf9-a611-7f376f949dc0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" Jul 28 16:03:35.805845 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:35.805792 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 16:03:35.805845 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:35.805840 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls podName:3345d4cb-2854-4bf9-a611-7f376f949dc0 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:39.805827139 +0000 UTC m=+87.839994443 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls") pod "cluster-samples-operator-6748489656-g5sp8" (UID: "3345d4cb-2854-4bf9-a611-7f376f949dc0") : secret "samples-operator-tls" not found Jul 28 16:03:37.412828 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:37.412793 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:37.514069 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:37.514037 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:37.921269 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:37.921237 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:37.921429 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:37.921303 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:37.921429 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:37.921398 2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 16:03:37.921558 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:37.921470 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle podName:6c1c9a74-abad-430a-ad3f-fe0d70833ca6 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:53.92145468 +0000 UTC m=+101.955621984 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle") pod "router-default-7f859bc589-qkbls" (UID: "6c1c9a74-abad-430a-ad3f-fe0d70833ca6") : configmap references non-existent config key: service-ca.crt Jul 28 16:03:37.921558 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:37.921487 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs podName:6c1c9a74-abad-430a-ad3f-fe0d70833ca6 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:53.921480652 +0000 UTC m=+101.955647955 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs") pod "router-default-7f859bc589-qkbls" (UID: "6c1c9a74-abad-430a-ad3f-fe0d70833ca6") : secret "router-metrics-certs-default" not found Jul 28 16:03:38.042164 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.042134 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:38.071151 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.071120 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-hwml6"] Jul 28 16:03:38.073618 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.073595 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hwml6" Jul 28 16:03:38.075690 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.075667 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 28 16:03:38.075772 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.075667 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qm5dn\"" Jul 28 16:03:38.076344 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.076325 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 28 16:03:38.085479 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.085453 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-hwml6"] Jul 28 16:03:38.123166 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.123144 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qkdw\" (UniqueName: \"kubernetes.io/projected/eae2599b-c9ae-4d7e-b5c6-7db936009ab8-kube-api-access-7qkdw\") pod \"migrator-6865c9967-hwml6\" (UID: \"eae2599b-c9ae-4d7e-b5c6-7db936009ab8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hwml6" Jul 28 16:03:38.138193 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.138162 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:38.223519 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.223463 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qkdw\" (UniqueName: \"kubernetes.io/projected/eae2599b-c9ae-4d7e-b5c6-7db936009ab8-kube-api-access-7qkdw\") pod \"migrator-6865c9967-hwml6\" (UID: \"eae2599b-c9ae-4d7e-b5c6-7db936009ab8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hwml6" Jul 28 16:03:38.232488 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.232458 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qkdw\" (UniqueName: \"kubernetes.io/projected/eae2599b-c9ae-4d7e-b5c6-7db936009ab8-kube-api-access-7qkdw\") pod \"migrator-6865c9967-hwml6\" (UID: \"eae2599b-c9ae-4d7e-b5c6-7db936009ab8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hwml6" Jul 28 16:03:38.238076 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.238054 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:38.383096 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.383076 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hwml6" Jul 28 16:03:38.505970 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.505903 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-hwml6"] Jul 28 16:03:38.509013 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:03:38.508986 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeae2599b_c9ae_4d7e_b5c6_7db936009ab8.slice/crio-2456db14b4a57e4c7f26401834971c451cdaae67eee391898018b705b084d232 WatchSource:0}: Error finding container 2456db14b4a57e4c7f26401834971c451cdaae67eee391898018b705b084d232: Status 404 returned error can't find the container with id 2456db14b4a57e4c7f26401834971c451cdaae67eee391898018b705b084d232 Jul 28 16:03:38.706351 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:38.706320 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hwml6" event={"ID":"eae2599b-c9ae-4d7e-b5c6-7db936009ab8","Type":"ContainerStarted","Data":"2456db14b4a57e4c7f26401834971c451cdaae67eee391898018b705b084d232"} Jul 28 16:03:39.709881 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:39.709854 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hwml6" event={"ID":"eae2599b-c9ae-4d7e-b5c6-7db936009ab8","Type":"ContainerStarted","Data":"9e29079ecd338fb14f5ff73149284552ef3fb36143dd63f5e6c94c0b50fb1e17"} Jul 28 16:03:39.762694 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:39.762672 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:39.836208 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:39.836182 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-g5sp8\" (UID: \"3345d4cb-2854-4bf9-a611-7f376f949dc0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" Jul 28 16:03:39.836301 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:39.836293 2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 16:03:39.836348 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:39.836337 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls podName:3345d4cb-2854-4bf9-a611-7f376f949dc0 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:47.836326078 +0000 UTC m=+95.870493381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls") pod "cluster-samples-operator-6748489656-g5sp8" (UID: "3345d4cb-2854-4bf9-a611-7f376f949dc0") : secret "samples-operator-tls" not found Jul 28 16:03:39.864758 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:39.864732 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:40.714738 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:40.714695 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hwml6" event={"ID":"eae2599b-c9ae-4d7e-b5c6-7db936009ab8","Type":"ContainerStarted","Data":"6954395a1e30db7778282c8b07678b096291f4e5d45ce9718eb49128c1bfb296"} Jul 28 16:03:40.736602 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:40.736550 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-hwml6" podStartSLOduration=1.624368938 podStartE2EDuration="2.736535421s" totalImagesPullingTime="1.112166483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:03:40.735392575 +0000 UTC m=+88.769559900" watchObservedRunningTime="2026-07-28 16:03:40.736535421 +0000 UTC m=+88.770702747" Jul 28 16:03:40.780730 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:40.780700 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:40.878298 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:40.878272 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:42.237673 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:42.237634 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:42.403716 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:42.403685 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:42.508958 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:42.508896 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:42.577117 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:42.577092 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:42.645329 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:42.645304 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:42.746619 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:42.746596 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:42.936286 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:42.936262 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:43.038123 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:43.038095 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:43.258778 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:43.258715 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:43.261301 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:43.261279 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:43.261361 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:43.261326 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:43.261460 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:43.261440 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 16:03:43.261516 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:03:43.261506 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls podName:6084bd94-c631-4f0f-a5a5-869566f1b430 nodeName:}" failed. No retries permitted until 2026-07-28 16:03:59.261491161 +0000 UTC m=+107.295658464 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8r9q6" (UID: "6084bd94-c631-4f0f-a5a5-869566f1b430") : secret "insights-runtime-extractor-tls" not found Jul 28 16:03:43.261860 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:43.261841 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6084bd94-c631-4f0f-a5a5-869566f1b430-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:43.528360 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:43.528293 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:44.135177 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:44.135147 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:44.733175 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:44.733142 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:44.927530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:44.927498 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:45.501965 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:45.501931 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:45.569040 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:45.569009 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:46.126972 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:46.126941 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:46.226176 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:46.226149 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:46.733359 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:46.733333 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:46.799804 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:46.799776 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:46.927107 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:46.927077 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:46.996636 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:46.996576 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:47.143628 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:47.143599 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:47.529877 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:47.529847 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:47.896143 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:47.896111 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-g5sp8\" (UID: \"3345d4cb-2854-4bf9-a611-7f376f949dc0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" Jul 28 16:03:47.898803 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:47.898771 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3345d4cb-2854-4bf9-a611-7f376f949dc0-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-g5sp8\" (UID: \"3345d4cb-2854-4bf9-a611-7f376f949dc0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" Jul 28 16:03:48.051176 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:48.051149 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" Jul 28 16:03:48.174625 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:48.174566 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8"] Jul 28 16:03:48.347561 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:48.347535 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:48.736870 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:48.736842 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:48.741831 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:48.741799 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" event={"ID":"3345d4cb-2854-4bf9-a611-7f376f949dc0","Type":"ContainerStarted","Data":"97ea7e45631de2d6f6f3f0a8ea571e638f56c1d91a3fb954ef895f3b0ea7bd31"} Jul 28 16:03:48.834467 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:48.834429 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:48.940243 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:48.940215 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:49.125711 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:49.125686 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:49.537650 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:49.537569 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:49.746665 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:49.746636 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" event={"ID":"3345d4cb-2854-4bf9-a611-7f376f949dc0","Type":"ContainerStarted","Data":"f27eb6763ebeeaad207a60e6e51e7b27ee3ec7e59d81fb0a6a8f4cc51a01d96d"} Jul 28 16:03:50.130735 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:50.130702 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:50.213680 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:50.213643 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:03:50.215913 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:50.215895 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4ecf78f-a024-4986-888d-67b82853a16b-metrics-tls\") pod \"dns-default-28lcc\" (UID: \"d4ecf78f-a024-4986-888d-67b82853a16b\") " pod="openshift-dns/dns-default-28lcc" Jul 28 16:03:50.270900 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:50.270877 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ssj24\"" Jul 28 16:03:50.278780 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:50.278760 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-28lcc" Jul 28 16:03:50.410175 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:50.410146 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-28lcc"] Jul 28 16:03:50.413574 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:03:50.413551 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4ecf78f_a024_4986_888d_67b82853a16b.slice/crio-c6035c7b211dffb8a23a28921bb56cb12e71b598e4f2136ff44c2e2f53a32606 WatchSource:0}: Error finding container c6035c7b211dffb8a23a28921bb56cb12e71b598e4f2136ff44c2e2f53a32606: Status 404 returned error can't find the container with id c6035c7b211dffb8a23a28921bb56cb12e71b598e4f2136ff44c2e2f53a32606 Jul 28 16:03:50.721471 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:50.721381 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:50.751660 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:50.751635 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" event={"ID":"3345d4cb-2854-4bf9-a611-7f376f949dc0","Type":"ContainerStarted","Data":"868566986bca5ecb6960ea589b4368d1339bc857905164924c347c879c293863"} Jul 28 16:03:50.752756 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:50.752735 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-28lcc" event={"ID":"d4ecf78f-a024-4986-888d-67b82853a16b","Type":"ContainerStarted","Data":"c6035c7b211dffb8a23a28921bb56cb12e71b598e4f2136ff44c2e2f53a32606"} Jul 28 16:03:50.768460 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:50.768393 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-g5sp8" podStartSLOduration=17.314414495 podStartE2EDuration="18.768379809s" totalImagesPullingTime="1.453965314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:03:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:03:50.768315818 +0000 UTC m=+98.802483143" watchObservedRunningTime="2026-07-28 16:03:50.768379809 +0000 UTC m=+98.802547135" Jul 28 16:03:52.760734 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:52.760699 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-28lcc" event={"ID":"d4ecf78f-a024-4986-888d-67b82853a16b","Type":"ContainerStarted","Data":"065b6a56158c445c9a4760bd15fefeb6319ef4238236e248572ef49a916abec4"} Jul 28 16:03:52.761087 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:52.760786 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-28lcc" event={"ID":"d4ecf78f-a024-4986-888d-67b82853a16b","Type":"ContainerStarted","Data":"9511742e59c8da8f20221601e50f1e06b4097d08e4f2ddb951789560279c1cb6"} Jul 28 16:03:52.761087 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:52.760816 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-28lcc" Jul 28 16:03:52.780247 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:52.780204 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-28lcc" podStartSLOduration=64.633476336 podStartE2EDuration="1m6.780190174s" totalImagesPullingTime="2.146713838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:02:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:03:52.779116373 +0000 UTC m=+100.813283698" watchObservedRunningTime="2026-07-28 16:03:52.780190174 +0000 UTC m=+100.814357501" Jul 28 16:03:53.437461 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:53.437428 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:53.543171 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:53.543136 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:53.938086 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:53.938061 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:53.938439 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:53.938123 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:53.938775 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:53.938745 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-service-ca-bundle\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:53.940492 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:53.940469 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c1c9a74-abad-430a-ad3f-fe0d70833ca6-metrics-certs\") pod \"router-default-7f859bc589-qkbls\" (UID: \"6c1c9a74-abad-430a-ad3f-fe0d70833ca6\") " pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:53.980079 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:53.980060 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:54.138666 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:54.138633 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7f859bc589-qkbls"] Jul 28 16:03:54.143921 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:03:54.143893 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c1c9a74_abad_430a_ad3f_fe0d70833ca6.slice/crio-0054d9cc86f02e3f1862ca327e961ea0cd1a1fe22e93365d14dd844acec88672 WatchSource:0}: Error finding container 0054d9cc86f02e3f1862ca327e961ea0cd1a1fe22e93365d14dd844acec88672: Status 404 returned error can't find the container with id 0054d9cc86f02e3f1862ca327e961ea0cd1a1fe22e93365d14dd844acec88672 Jul 28 16:03:54.212174 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:54.212149 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:54.671361 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:54.671337 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hxxbk" Jul 28 16:03:54.767584 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:54.767556 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f859bc589-qkbls" event={"ID":"6c1c9a74-abad-430a-ad3f-fe0d70833ca6","Type":"ContainerStarted","Data":"880c8c883c53e9d3c9ab444b89c0bd973ea710086a52f092925bf00712ca4a73"} Jul 28 16:03:54.767687 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:54.767598 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7f859bc589-qkbls" event={"ID":"6c1c9a74-abad-430a-ad3f-fe0d70833ca6","Type":"ContainerStarted","Data":"0054d9cc86f02e3f1862ca327e961ea0cd1a1fe22e93365d14dd844acec88672"} Jul 28 16:03:54.789688 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:54.789645 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7f859bc589-qkbls" podStartSLOduration=32.789631573 podStartE2EDuration="32.789631573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:03:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:03:54.788683143 +0000 UTC m=+102.822850469" watchObservedRunningTime="2026-07-28 16:03:54.789631573 +0000 UTC m=+102.823798899" Jul 28 16:03:54.981062 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:54.981007 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:54.983249 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:54.983230 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:55.771183 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:55.771149 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:55.772458 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:55.772435 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7f859bc589-qkbls" Jul 28 16:03:56.792122 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:56.792088 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:56.892366 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:56.892334 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:56.992237 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:56.992210 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:57.185050 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:57.185024 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:59.273794 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:59.273763 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:59.276071 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:59.276040 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6084bd94-c631-4f0f-a5a5-869566f1b430-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8r9q6\" (UID: \"6084bd94-c631-4f0f-a5a5-869566f1b430\") " pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:59.346665 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:59.346640 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8r9q6" Jul 28 16:03:59.474201 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:59.474172 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8r9q6"] Jul 28 16:03:59.477326 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:03:59.477297 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6084bd94_c631_4f0f_a5a5_869566f1b430.slice/crio-b1c8c52d118d58798df38d0369711016963c5ea651e24e318d439ac2c92fce76 WatchSource:0}: Error finding container b1c8c52d118d58798df38d0369711016963c5ea651e24e318d439ac2c92fce76: Status 404 returned error can't find the container with id b1c8c52d118d58798df38d0369711016963c5ea651e24e318d439ac2c92fce76 Jul 28 16:03:59.648540 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:59.648517 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:59.753082 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:59.753055 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:03:59.784046 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:03:59.784016 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8r9q6" event={"ID":"6084bd94-c631-4f0f-a5a5-869566f1b430","Type":"ContainerStarted","Data":"b1c8c52d118d58798df38d0369711016963c5ea651e24e318d439ac2c92fce76"} Jul 28 16:04:02.212635 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:02.212604 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:02.766017 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:02.765983 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-28lcc" Jul 28 16:04:04.771654 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:04.771626 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-c77sp"] Jul 28 16:04:04.775065 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:04.774763 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c77sp" Jul 28 16:04:04.776965 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:04:04.776841 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"network-diagnostics-dockercfg-8h27l\" is forbidden: User \"system:node:ip-10-0-138-10.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-diagnostics\": no relationship found between node 'ip-10-0-138-10.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-8h27l\"" type="*v1.Secret" Jul 28 16:04:04.776965 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:04:04.776925 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"network-check-source-7d88cb55d8-c77sp\" is forbidden: User \"system:node:ip-10-0-138-10.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-network-diagnostics\": no relationship found between node 'ip-10-0-138-10.ec2.internal' and this object" podUID="483f5d37-caad-46a6-9857-a10eb7e50c30" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c77sp" Jul 28 16:04:04.784919 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:04.784868 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-c77sp"] Jul 28 16:04:04.803678 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:04.803648 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8r9q6" event={"ID":"6084bd94-c631-4f0f-a5a5-869566f1b430","Type":"ContainerStarted","Data":"df4d1b09e676992b6c3d6df0ab5d2a419917ff88536a9bef8b254d5521144bf6"} Jul 28 16:04:04.885923 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:04.885891 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-95fz5"] Jul 28 16:04:04.889087 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:04.889069 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-95fz5" Jul 28 16:04:04.893197 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:04.892982 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-l6jzg\"" Jul 28 16:04:04.896533 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:04.896512 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 28 16:04:04.907666 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:04.907641 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-95fz5"] Jul 28 16:04:04.919635 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:04.919608 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqj89\" (UniqueName: \"kubernetes.io/projected/483f5d37-caad-46a6-9857-a10eb7e50c30-kube-api-access-xqj89\") pod \"network-check-source-7d88cb55d8-c77sp\" (UID: \"483f5d37-caad-46a6-9857-a10eb7e50c30\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c77sp" Jul 28 16:04:05.016363 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.016287 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:05.020451 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.020402 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f377f0bf-3bdd-41f5-bedc-b5259079ff6f-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-95fz5\" (UID: \"f377f0bf-3bdd-41f5-bedc-b5259079ff6f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-95fz5" Jul 28 16:04:05.020556 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.020509 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqj89\" (UniqueName: \"kubernetes.io/projected/483f5d37-caad-46a6-9857-a10eb7e50c30-kube-api-access-xqj89\") pod \"network-check-source-7d88cb55d8-c77sp\" (UID: \"483f5d37-caad-46a6-9857-a10eb7e50c30\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c77sp" Jul 28 16:04:05.058910 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.058861 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqj89\" (UniqueName: \"kubernetes.io/projected/483f5d37-caad-46a6-9857-a10eb7e50c30-kube-api-access-xqj89\") pod \"network-check-source-7d88cb55d8-c77sp\" (UID: \"483f5d37-caad-46a6-9857-a10eb7e50c30\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c77sp" Jul 28 16:04:05.106590 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.106564 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-757d4dcc58-mqzbm"] Jul 28 16:04:05.109469 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.109452 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.109919 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.109890 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:05.115700 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.115679 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 28 16:04:05.115796 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.115764 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8qmb9\"" Jul 28 16:04:05.116017 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.115998 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 28 16:04:05.116346 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.116324 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 28 16:04:05.120865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.120844 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f377f0bf-3bdd-41f5-bedc-b5259079ff6f-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-95fz5\" (UID: \"f377f0bf-3bdd-41f5-bedc-b5259079ff6f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-95fz5" Jul 28 16:04:05.121384 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.121365 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 28 16:04:05.123472 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.123452 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f377f0bf-3bdd-41f5-bedc-b5259079ff6f-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-95fz5\" (UID: \"f377f0bf-3bdd-41f5-bedc-b5259079ff6f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-95fz5" Jul 28 16:04:05.128793 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.128773 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-757d4dcc58-mqzbm"] Jul 28 16:04:05.201005 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.200982 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-95fz5" Jul 28 16:04:05.221300 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.221271 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eca560de-d0ef-4627-81db-12d0636ced3f-image-registry-private-configuration\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.221420 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.221325 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eca560de-d0ef-4627-81db-12d0636ced3f-trusted-ca\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.221420 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.221361 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eca560de-d0ef-4627-81db-12d0636ced3f-registry-certificates\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.221539 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.221442 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eca560de-d0ef-4627-81db-12d0636ced3f-registry-tls\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.221539 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.221483 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eca560de-d0ef-4627-81db-12d0636ced3f-installation-pull-secrets\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.221636 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.221553 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gtvg\" (UniqueName: \"kubernetes.io/projected/eca560de-d0ef-4627-81db-12d0636ced3f-kube-api-access-4gtvg\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.221636 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.221585 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eca560de-d0ef-4627-81db-12d0636ced3f-bound-sa-token\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.221636 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.221615 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eca560de-d0ef-4627-81db-12d0636ced3f-ca-trust-extracted\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.321983 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.321943 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eca560de-d0ef-4627-81db-12d0636ced3f-image-registry-private-configuration\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.322125 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.322006 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eca560de-d0ef-4627-81db-12d0636ced3f-trusted-ca\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.322125 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.322094 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eca560de-d0ef-4627-81db-12d0636ced3f-registry-certificates\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.322241 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.322139 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eca560de-d0ef-4627-81db-12d0636ced3f-registry-tls\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.322241 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.322171 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eca560de-d0ef-4627-81db-12d0636ced3f-installation-pull-secrets\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.322241 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.322233 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gtvg\" (UniqueName: \"kubernetes.io/projected/eca560de-d0ef-4627-81db-12d0636ced3f-kube-api-access-4gtvg\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.323037 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.322445 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eca560de-d0ef-4627-81db-12d0636ced3f-bound-sa-token\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.323037 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.322659 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eca560de-d0ef-4627-81db-12d0636ced3f-ca-trust-extracted\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.323037 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.323004 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eca560de-d0ef-4627-81db-12d0636ced3f-ca-trust-extracted\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.323220 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.323123 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eca560de-d0ef-4627-81db-12d0636ced3f-registry-certificates\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.323679 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.323596 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eca560de-d0ef-4627-81db-12d0636ced3f-trusted-ca\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.325072 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.324952 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eca560de-d0ef-4627-81db-12d0636ced3f-registry-tls\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.325072 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.324997 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eca560de-d0ef-4627-81db-12d0636ced3f-image-registry-private-configuration\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.325249 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.325232 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eca560de-d0ef-4627-81db-12d0636ced3f-installation-pull-secrets\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.340179 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.340156 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gtvg\" (UniqueName: \"kubernetes.io/projected/eca560de-d0ef-4627-81db-12d0636ced3f-kube-api-access-4gtvg\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.347694 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.347669 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eca560de-d0ef-4627-81db-12d0636ced3f-bound-sa-token\") pod \"image-registry-757d4dcc58-mqzbm\" (UID: \"eca560de-d0ef-4627-81db-12d0636ced3f\") " pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.348891 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.348868 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-95fz5"] Jul 28 16:04:05.352583 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:04:05.352559 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf377f0bf_3bdd_41f5_bedc_b5259079ff6f.slice/crio-ebb6a11e7b8693fcfaa57355e3c82e59d917f63bfac2fe03436be94acd72f800 WatchSource:0}: Error finding container ebb6a11e7b8693fcfaa57355e3c82e59d917f63bfac2fe03436be94acd72f800: Status 404 returned error can't find the container with id ebb6a11e7b8693fcfaa57355e3c82e59d917f63bfac2fe03436be94acd72f800 Jul 28 16:04:05.420555 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.420532 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:05.570529 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.570495 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-757d4dcc58-mqzbm"] Jul 28 16:04:05.576059 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:04:05.576035 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca560de_d0ef_4627_81db_12d0636ced3f.slice/crio-9ae3a541e5b1cc9b0e59341c3736e2b64c506a5f243c9747ca9eb213adffd37f WatchSource:0}: Error finding container 9ae3a541e5b1cc9b0e59341c3736e2b64c506a5f243c9747ca9eb213adffd37f: Status 404 returned error can't find the container with id 9ae3a541e5b1cc9b0e59341c3736e2b64c506a5f243c9747ca9eb213adffd37f Jul 28 16:04:05.808214 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.808176 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-95fz5" event={"ID":"f377f0bf-3bdd-41f5-bedc-b5259079ff6f","Type":"ContainerStarted","Data":"ebb6a11e7b8693fcfaa57355e3c82e59d917f63bfac2fe03436be94acd72f800"} Jul 28 16:04:05.809347 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:05.809316 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" event={"ID":"eca560de-d0ef-4627-81db-12d0636ced3f","Type":"ContainerStarted","Data":"9ae3a541e5b1cc9b0e59341c3736e2b64c506a5f243c9747ca9eb213adffd37f"} Jul 28 16:04:06.076644 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:06.076611 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-8h27l\"" Jul 28 16:04:06.076807 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:06.076676 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c77sp" Jul 28 16:04:06.718394 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:06.718366 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:06.887445 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:06.887416 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:06.946391 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:06.946359 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:07.607659 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:07.607628 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-c77sp"] Jul 28 16:04:07.791568 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:04:07.791495 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod483f5d37_caad_46a6_9857_a10eb7e50c30.slice/crio-45c70f2959167f46913d8bfc6f82a90fc2532881439f78098e7bdb45508f66ad WatchSource:0}: Error finding container 45c70f2959167f46913d8bfc6f82a90fc2532881439f78098e7bdb45508f66ad: Status 404 returned error can't find the container with id 45c70f2959167f46913d8bfc6f82a90fc2532881439f78098e7bdb45508f66ad Jul 28 16:04:07.817221 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:07.817194 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" event={"ID":"eca560de-d0ef-4627-81db-12d0636ced3f","Type":"ContainerStarted","Data":"9350c937474c42e8145db887355e3a7628b8e1937fa6862862a954d2892af910"} Jul 28 16:04:07.817335 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:07.817253 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:07.818231 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:07.818213 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c77sp" event={"ID":"483f5d37-caad-46a6-9857-a10eb7e50c30","Type":"ContainerStarted","Data":"45c70f2959167f46913d8bfc6f82a90fc2532881439f78098e7bdb45508f66ad"} Jul 28 16:04:07.838059 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:07.838019 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" podStartSLOduration=2.838004293 podStartE2EDuration="2.838004293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:04:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:04:07.837314159 +0000 UTC m=+115.871481486" watchObservedRunningTime="2026-07-28 16:04:07.838004293 +0000 UTC m=+115.872171653" Jul 28 16:04:08.824402 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:08.824364 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8r9q6" event={"ID":"6084bd94-c631-4f0f-a5a5-869566f1b430","Type":"ContainerStarted","Data":"775ac3fe0afb638cc9617c76db916f5730b3ddcec0fcbfb7efcbc24daa2eece8"} Jul 28 16:04:08.826064 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:08.825997 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c77sp" event={"ID":"483f5d37-caad-46a6-9857-a10eb7e50c30","Type":"ContainerStarted","Data":"09e28b104b859a8b8b3fdc09ef46c9c02602d755a1b0f082bd01552332442856"} Jul 28 16:04:08.827753 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:08.827724 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-95fz5" event={"ID":"f377f0bf-3bdd-41f5-bedc-b5259079ff6f","Type":"ContainerStarted","Data":"bbaaf10d94bc6740444e307b8a27737e716e0314164c639b50df623483c54e84"} Jul 28 16:04:08.846994 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:08.846953 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-c77sp" podStartSLOduration=4.846941149 podStartE2EDuration="4.846941149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:04:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:04:08.845466033 +0000 UTC m=+116.879633360" watchObservedRunningTime="2026-07-28 16:04:08.846941149 +0000 UTC m=+116.881108474" Jul 28 16:04:08.863523 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:08.863473 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-95fz5" podStartSLOduration=2.378918755 podStartE2EDuration="4.86345772s" totalImagesPullingTime="2.484538965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:04:08.861983155 +0000 UTC m=+116.896150483" watchObservedRunningTime="2026-07-28 16:04:08.86345772 +0000 UTC m=+116.897625049" Jul 28 16:04:09.528686 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:09.528651 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:09.595450 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:09.595418 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:09.831302 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:09.831268 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-95fz5" Jul 28 16:04:09.836584 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:09.836556 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-95fz5" Jul 28 16:04:09.981906 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:09.981864 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:10.090232 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:10.090146 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:10.174894 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:10.174861 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:10.631773 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:10.631743 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:10.776746 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:10.776711 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:10.876218 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:10.876178 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:11.840646 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:11.840604 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8r9q6" event={"ID":"6084bd94-c631-4f0f-a5a5-869566f1b430","Type":"ContainerStarted","Data":"78f544c0b35696651b151ff5330e33b81583d811b006a2a2562301919993d072"} Jul 28 16:04:11.861722 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:11.861585 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8r9q6" podStartSLOduration=33.064415062 podStartE2EDuration="44.861564297s" totalImagesPullingTime="11.797149235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:03:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:04:11.86101269 +0000 UTC m=+119.895180014" watchObservedRunningTime="2026-07-28 16:04:11.861564297 +0000 UTC m=+119.895731626" Jul 28 16:04:13.063392 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:13.063361 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:13.163576 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:13.163546 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:13.862939 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:13.862904 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:13.928084 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:13.928052 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:14.585926 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:14.585896 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:14.710657 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:14.710624 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:16.212446 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:16.212401 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:18.255343 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.255311 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:18.646156 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.646129 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:18.713908 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.713881 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-m8bzl"] Jul 28 16:04:18.718972 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.718954 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.720843 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.720822 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 16:04:18.720962 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.720946 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 16:04:18.721480 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.721466 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 16:04:18.721524 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.721501 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 16:04:18.722104 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.722088 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5dx87\"" Jul 28 16:04:18.722168 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.722113 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 16:04:18.722211 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.722095 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 16:04:18.728698 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.727207 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7"] Jul 28 16:04:18.731737 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.731719 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.733590 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.733572 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 28 16:04:18.733884 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.733871 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 28 16:04:18.734126 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.734114 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-c6js5\"" Jul 28 16:04:18.734599 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.734585 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 28 16:04:18.741301 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.741282 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7"] Jul 28 16:04:18.761752 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.761726 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:18.821668 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.821643 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxt75\" (UniqueName: \"kubernetes.io/projected/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-kube-api-access-xxt75\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.821758 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.821671 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f7c7d70-020c-493d-987d-ec6e3291368d-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.821758 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.821689 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.821758 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.821721 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-textfile\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.821758 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.821737 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2f7c7d70-020c-493d-987d-ec6e3291368d-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.821949 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.821786 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.821949 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.821818 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-tls\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.821949 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.821855 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qscl\" (UniqueName: \"kubernetes.io/projected/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-api-access-6qscl\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.821949 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.821905 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-wtmp\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.822135 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.821964 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-accelerators-collector-config\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.822135 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.822005 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-sys\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.822135 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.822035 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-root\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.822135 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.822057 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-metrics-client-ca\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.822135 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.822082 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.822135 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.822116 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.865957 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.865935 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:18.923038 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.922993 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxt75\" (UniqueName: \"kubernetes.io/projected/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-kube-api-access-xxt75\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.923038 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923016 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f7c7d70-020c-493d-987d-ec6e3291368d-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.923038 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923035 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.923200 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923056 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-textfile\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.923200 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923080 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2f7c7d70-020c-493d-987d-ec6e3291368d-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.923281 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923201 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.923281 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923246 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-tls\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.923362 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923286 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qscl\" (UniqueName: \"kubernetes.io/projected/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-api-access-6qscl\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.923362 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:04:18.923325 2550 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 28 16:04:18.923362 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923329 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-wtmp\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.923530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923383 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-accelerators-collector-config\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.923530 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:04:18.923393 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-state-metrics-tls podName:2f7c7d70-020c-493d-987d-ec6e3291368d nodeName:}" failed. No retries permitted until 2026-07-28 16:04:19.423377117 +0000 UTC m=+127.457544441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-lx4v7" (UID: "2f7c7d70-020c-493d-987d-ec6e3291368d") : secret "kube-state-metrics-tls" not found Jul 28 16:04:18.923530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923427 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-textfile\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.923530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923448 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-sys\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.923530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923491 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-root\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.923530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923498 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2f7c7d70-020c-493d-987d-ec6e3291368d-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.923530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923517 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-metrics-client-ca\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.923889 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923494 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-sys\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.923889 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923571 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-root\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.923889 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923657 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-wtmp\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.923889 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923681 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.923889 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923721 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.923889 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923795 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f7c7d70-020c-493d-987d-ec6e3291368d-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.924261 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.923978 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-accelerators-collector-config\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.924261 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.924066 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-metrics-client-ca\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.924373 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.924326 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.925814 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.925785 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-tls\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.925909 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.925855 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.925909 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.925869 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.932014 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.931968 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxt75\" (UniqueName: \"kubernetes.io/projected/8b930b8e-8a7a-4045-8bc9-2e45b0b3c073-kube-api-access-xxt75\") pod \"node-exporter-m8bzl\" (UID: \"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073\") " pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:18.932118 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.932099 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qscl\" (UniqueName: \"kubernetes.io/projected/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-api-access-6qscl\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:18.963241 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:18.963215 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:19.032589 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:19.032569 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m8bzl" Jul 28 16:04:19.045048 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:04:19.045024 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b930b8e_8a7a_4045_8bc9_2e45b0b3c073.slice/crio-52731c223c37bad4bb302bdc9276d2c2f252eb6a5127547a93b7bf566be3f280 WatchSource:0}: Error finding container 52731c223c37bad4bb302bdc9276d2c2f252eb6a5127547a93b7bf566be3f280: Status 404 returned error can't find the container with id 52731c223c37bad4bb302bdc9276d2c2f252eb6a5127547a93b7bf566be3f280 Jul 28 16:04:19.058390 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:19.058365 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:19.160192 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:19.160171 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:19.352672 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:19.352652 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:19.427076 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:19.427053 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:19.429225 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:19.429195 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f7c7d70-020c-493d-987d-ec6e3291368d-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-lx4v7\" (UID: \"2f7c7d70-020c-493d-987d-ec6e3291368d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:19.450903 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:19.450875 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:19.564145 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:19.564106 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:19.640142 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:19.640070 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" Jul 28 16:04:19.689670 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:19.689640 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:19.781591 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:19.781552 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:19.788774 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:19.788751 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7"] Jul 28 16:04:19.867969 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:19.867936 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m8bzl" event={"ID":"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073","Type":"ContainerStarted","Data":"52731c223c37bad4bb302bdc9276d2c2f252eb6a5127547a93b7bf566be3f280"} Jul 28 16:04:19.878368 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:19.878343 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:19.962080 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:04:19.962022 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f7c7d70_020c_493d_987d_ec6e3291368d.slice/crio-90136902b3a1aa7dae10c2eefc674effbd6abf43f8a41658c9faf5b1612363ce WatchSource:0}: Error finding container 90136902b3a1aa7dae10c2eefc674effbd6abf43f8a41658c9faf5b1612363ce: Status 404 returned error can't find the container with id 90136902b3a1aa7dae10c2eefc674effbd6abf43f8a41658c9faf5b1612363ce Jul 28 16:04:20.606734 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.606706 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct"] Jul 28 16:04:20.610303 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.610277 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.612820 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.612776 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 28 16:04:20.612944 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.612843 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 28 16:04:20.613227 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.613072 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-h44nt\"" Jul 28 16:04:20.613227 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.613088 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 28 16:04:20.613227 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.613105 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 28 16:04:20.613227 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.613107 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2q3tckfeic7t7\"" Jul 28 16:04:20.613227 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.613134 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 28 16:04:20.622272 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.622252 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct"] Jul 28 16:04:20.738373 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.738344 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.738531 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.738387 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-tls\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.738531 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.738457 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3616399a-0634-4f02-90ef-58950b10598b-metrics-client-ca\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.738531 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.738485 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbbxj\" (UniqueName: \"kubernetes.io/projected/3616399a-0634-4f02-90ef-58950b10598b-kube-api-access-pbbxj\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.738687 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.738566 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-grpc-tls\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.738687 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.738609 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.738687 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.738646 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.738823 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.738725 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.839737 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.839708 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3616399a-0634-4f02-90ef-58950b10598b-metrics-client-ca\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.839857 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.839748 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbbxj\" (UniqueName: \"kubernetes.io/projected/3616399a-0634-4f02-90ef-58950b10598b-kube-api-access-pbbxj\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.839857 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.839784 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-grpc-tls\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.839857 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.839816 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.839857 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.839844 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.840072 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.839910 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.840072 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.839940 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.840072 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.839967 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-tls\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.840568 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.840520 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3616399a-0634-4f02-90ef-58950b10598b-metrics-client-ca\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.843128 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.843082 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.843240 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.843158 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-tls\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.843240 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.843220 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.843552 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.843512 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.843791 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.843766 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.843924 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.843900 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3616399a-0634-4f02-90ef-58950b10598b-secret-grpc-tls\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.850239 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.850220 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbbxj\" (UniqueName: \"kubernetes.io/projected/3616399a-0634-4f02-90ef-58950b10598b-kube-api-access-pbbxj\") pod \"thanos-querier-6c878fcd8d-tp2ct\" (UID: \"3616399a-0634-4f02-90ef-58950b10598b\") " pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:20.873565 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.873516 2550 generic.go:356] "Generic (PLEG): container finished" podID="8b930b8e-8a7a-4045-8bc9-2e45b0b3c073" containerID="8832c7fc17dd4553f500b35ec5d971c7d36cc368c7a84ae752ced81dec6792fd" exitCode=0 Jul 28 16:04:20.873638 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.873588 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m8bzl" event={"ID":"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073","Type":"ContainerDied","Data":"8832c7fc17dd4553f500b35ec5d971c7d36cc368c7a84ae752ced81dec6792fd"} Jul 28 16:04:20.874749 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.874730 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" event={"ID":"2f7c7d70-020c-493d-987d-ec6e3291368d","Type":"ContainerStarted","Data":"90136902b3a1aa7dae10c2eefc674effbd6abf43f8a41658c9faf5b1612363ce"} Jul 28 16:04:20.921001 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:20.920978 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:21.320496 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:21.320381 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct"] Jul 28 16:04:21.323098 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:04:21.323068 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3616399a_0634_4f02_90ef_58950b10598b.slice/crio-25a2f0bf8da7a15d1f25e3be3cee531eb0049ef15e9e99d0eea57fcc59dc1de1 WatchSource:0}: Error finding container 25a2f0bf8da7a15d1f25e3be3cee531eb0049ef15e9e99d0eea57fcc59dc1de1: Status 404 returned error can't find the container with id 25a2f0bf8da7a15d1f25e3be3cee531eb0049ef15e9e99d0eea57fcc59dc1de1 Jul 28 16:04:21.505100 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:21.505066 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:21.562967 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:21.562939 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:21.658108 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:21.658071 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:21.813509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:21.813478 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:21.881363 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:21.881326 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m8bzl" event={"ID":"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073","Type":"ContainerStarted","Data":"75db8a82008f00d75d4fb04e86af10610a2279c574883d3cfbf8fda64ad4a3c8"} Jul 28 16:04:21.881544 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:21.881393 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m8bzl" event={"ID":"8b930b8e-8a7a-4045-8bc9-2e45b0b3c073","Type":"ContainerStarted","Data":"1788e2979f7af8ee96acb30b89be2ecb4572141e3fa148928598fe20a925d723"} Jul 28 16:04:21.884092 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:21.884061 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" event={"ID":"2f7c7d70-020c-493d-987d-ec6e3291368d","Type":"ContainerStarted","Data":"f74b834934e75351fa94b7f151aeee624820633c102e03e49f4f9e601517c2f4"} Jul 28 16:04:21.884218 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:21.884124 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" event={"ID":"2f7c7d70-020c-493d-987d-ec6e3291368d","Type":"ContainerStarted","Data":"a0479b99a4e75812e7822cf5215cd0fc05f89b945e91414ef9ebe4723d501663"} Jul 28 16:04:21.884218 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:21.884156 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" event={"ID":"2f7c7d70-020c-493d-987d-ec6e3291368d","Type":"ContainerStarted","Data":"a3d5607f0a16f17b1785008b007f252c6542fe9c6a6ebce39e97ba9a5d080d6e"} Jul 28 16:04:21.885614 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:21.885588 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" event={"ID":"3616399a-0634-4f02-90ef-58950b10598b","Type":"ContainerStarted","Data":"25a2f0bf8da7a15d1f25e3be3cee531eb0049ef15e9e99d0eea57fcc59dc1de1"} Jul 28 16:04:21.902705 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:21.902664 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-m8bzl" podStartSLOduration=2.883485286 podStartE2EDuration="3.902647704s" totalImagesPullingTime="959.012807ms" totalInitContainerRuntime="60.149611ms" isStatefulPod=true podCreationTimestamp="2026-07-28 16:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:04:21.901434836 +0000 UTC m=+129.935602163" watchObservedRunningTime="2026-07-28 16:04:21.902647704 +0000 UTC m=+129.936815033" Jul 28 16:04:21.922788 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:21.922751 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-lx4v7" podStartSLOduration=2.653028199 podStartE2EDuration="3.922738564s" totalImagesPullingTime="1.269710365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:04:21.921901508 +0000 UTC m=+129.956068834" watchObservedRunningTime="2026-07-28 16:04:21.922738564 +0000 UTC m=+129.956905890" Jul 28 16:04:22.150695 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:22.150625 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs\") pod \"network-metrics-daemon-ngzg7\" (UID: \"ddb31891-1896-4c34-97c8-6e45e42d349c\") " pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:04:22.153005 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:22.152980 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddb31891-1896-4c34-97c8-6e45e42d349c-metrics-certs\") pod \"network-metrics-daemon-ngzg7\" (UID: \"ddb31891-1896-4c34-97c8-6e45e42d349c\") " pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:04:22.223646 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:22.223617 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:22.301284 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:22.301254 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:22.449788 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:22.449725 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f7pxp\"" Jul 28 16:04:22.457890 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:22.457863 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ngzg7" Jul 28 16:04:22.590438 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:22.590350 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ngzg7"] Jul 28 16:04:22.594187 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:04:22.594157 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb31891_1896_4c34_97c8_6e45e42d349c.slice/crio-c84790c6b0f6d1b551d9476c290874e028e52b2ceb11a947d4a232462d085e7d WatchSource:0}: Error finding container c84790c6b0f6d1b551d9476c290874e028e52b2ceb11a947d4a232462d085e7d: Status 404 returned error can't find the container with id c84790c6b0f6d1b551d9476c290874e028e52b2ceb11a947d4a232462d085e7d Jul 28 16:04:22.890755 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:22.890712 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ngzg7" event={"ID":"ddb31891-1896-4c34-97c8-6e45e42d349c","Type":"ContainerStarted","Data":"c84790c6b0f6d1b551d9476c290874e028e52b2ceb11a947d4a232462d085e7d"} Jul 28 16:04:23.218923 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.218792 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:23.322616 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.322590 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:23.334585 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.334558 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk"] Jul 28 16:04:23.337522 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.337506 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" Jul 28 16:04:23.339638 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.339618 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 28 16:04:23.339739 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.339714 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xbg7k\"" Jul 28 16:04:23.346670 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.346646 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk"] Jul 28 16:04:23.389202 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.389141 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:23.461576 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.461548 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f26a3188-b900-4f3a-88d3-2f9a856e48b7-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-xn5jk\" (UID: \"f26a3188-b900-4f3a-88d3-2f9a856e48b7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" Jul 28 16:04:23.461696 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.461607 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skp7h\" (UniqueName: \"kubernetes.io/projected/f26a3188-b900-4f3a-88d3-2f9a856e48b7-kube-api-access-skp7h\") pod \"monitoring-plugin-68b7fc9574-xn5jk\" (UID: \"f26a3188-b900-4f3a-88d3-2f9a856e48b7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" Jul 28 16:04:23.562714 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.562685 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skp7h\" (UniqueName: \"kubernetes.io/projected/f26a3188-b900-4f3a-88d3-2f9a856e48b7-kube-api-access-skp7h\") pod \"monitoring-plugin-68b7fc9574-xn5jk\" (UID: \"f26a3188-b900-4f3a-88d3-2f9a856e48b7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" Jul 28 16:04:23.562851 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.562813 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f26a3188-b900-4f3a-88d3-2f9a856e48b7-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-xn5jk\" (UID: \"f26a3188-b900-4f3a-88d3-2f9a856e48b7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" Jul 28 16:04:23.562975 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:04:23.562956 2550 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 28 16:04:23.563037 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:04:23.563024 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f26a3188-b900-4f3a-88d3-2f9a856e48b7-monitoring-plugin-cert podName:f26a3188-b900-4f3a-88d3-2f9a856e48b7 nodeName:}" failed. No retries permitted until 2026-07-28 16:04:24.06300353 +0000 UTC m=+132.097170840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/f26a3188-b900-4f3a-88d3-2f9a856e48b7-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-xn5jk" (UID: "f26a3188-b900-4f3a-88d3-2f9a856e48b7") : secret "monitoring-plugin-cert" not found Jul 28 16:04:23.576422 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.576383 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skp7h\" (UniqueName: \"kubernetes.io/projected/f26a3188-b900-4f3a-88d3-2f9a856e48b7-kube-api-access-skp7h\") pod \"monitoring-plugin-68b7fc9574-xn5jk\" (UID: \"f26a3188-b900-4f3a-88d3-2f9a856e48b7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" Jul 28 16:04:23.764213 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.764093 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:23.896290 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.896258 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" event={"ID":"3616399a-0634-4f02-90ef-58950b10598b","Type":"ContainerStarted","Data":"5dfdcd9bc6bfb4e28614d570e6235bc6c479843d31f59b59c9b5b666b12f3f76"} Jul 28 16:04:23.896635 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.896326 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" event={"ID":"3616399a-0634-4f02-90ef-58950b10598b","Type":"ContainerStarted","Data":"51b5eee431bc722cd1b62a3cfedc630dcc6a41d9f4be5f09bd3b045d832cf396"} Jul 28 16:04:23.896635 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:23.896365 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" event={"ID":"3616399a-0634-4f02-90ef-58950b10598b","Type":"ContainerStarted","Data":"20910fc67c3f0a05f4d758b5ae51651248d45643ee6d6b7badc0fa3ec7a8fdc6"} Jul 28 16:04:24.066593 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.066563 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f26a3188-b900-4f3a-88d3-2f9a856e48b7-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-xn5jk\" (UID: \"f26a3188-b900-4f3a-88d3-2f9a856e48b7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" Jul 28 16:04:24.068661 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.068644 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f26a3188-b900-4f3a-88d3-2f9a856e48b7-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-xn5jk\" (UID: \"f26a3188-b900-4f3a-88d3-2f9a856e48b7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" Jul 28 16:04:24.263790 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.263765 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" Jul 28 16:04:24.347204 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.347126 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:24.408399 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.408310 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk"] Jul 28 16:04:24.411573 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:04:24.411504 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf26a3188_b900_4f3a_88d3_2f9a856e48b7.slice/crio-4c8aac28034b7ae52bb739b2c7bac97bcdd188c291ec0a09ceea0f2dc8c1d770 WatchSource:0}: Error finding container 4c8aac28034b7ae52bb739b2c7bac97bcdd188c291ec0a09ceea0f2dc8c1d770: Status 404 returned error can't find the container with id 4c8aac28034b7ae52bb739b2c7bac97bcdd188c291ec0a09ceea0f2dc8c1d770 Jul 28 16:04:24.425527 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.425501 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:24.566062 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.566037 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84945dcd9-hzqsj"] Jul 28 16:04:24.569452 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.569395 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.571994 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.571974 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-d4kl9\"" Jul 28 16:04:24.572214 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.572190 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 28 16:04:24.572564 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.572545 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 28 16:04:24.572646 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.572630 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 28 16:04:24.573050 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.573025 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 28 16:04:24.573528 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.573508 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 28 16:04:24.573707 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.573687 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 28 16:04:24.575710 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.573915 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 28 16:04:24.577688 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.577659 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 28 16:04:24.582525 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.582501 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84945dcd9-hzqsj"] Jul 28 16:04:24.671708 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.671679 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-config\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.671795 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.671726 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-oauth-config\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.671795 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.671763 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z74z5\" (UniqueName: \"kubernetes.io/projected/2ca9ca85-d92d-402b-b518-37c0f5d157fc-kube-api-access-z74z5\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.671866 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.671793 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-service-ca\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.671866 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.671850 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-oauth-serving-cert\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.671938 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.671897 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-serving-cert\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.671938 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.671922 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-trusted-ca-bundle\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.773135 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.773112 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-service-ca\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.773239 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.773147 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-oauth-serving-cert\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.773239 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.773173 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-serving-cert\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.773239 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.773204 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-trusted-ca-bundle\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.773398 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.773250 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-config\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.773398 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.773281 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-oauth-config\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.773398 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.773314 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z74z5\" (UniqueName: \"kubernetes.io/projected/2ca9ca85-d92d-402b-b518-37c0f5d157fc-kube-api-access-z74z5\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.773852 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.773826 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-service-ca\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.773852 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.773844 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-oauth-serving-cert\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.774014 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.773982 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-config\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.774177 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.774155 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-trusted-ca-bundle\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.775532 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.775513 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-serving-cert\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.775532 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.775519 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-oauth-config\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.783086 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.783064 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z74z5\" (UniqueName: \"kubernetes.io/projected/2ca9ca85-d92d-402b-b518-37c0f5d157fc-kube-api-access-z74z5\") pod \"console-84945dcd9-hzqsj\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.862337 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.862273 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:24.882366 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.882341 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:24.904069 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.904046 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" event={"ID":"3616399a-0634-4f02-90ef-58950b10598b","Type":"ContainerStarted","Data":"3ab46f7896604ead7be4edfd8af79b95c7a56a8660ada2772d992fb2e7b41695"} Jul 28 16:04:24.904450 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.904147 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" event={"ID":"3616399a-0634-4f02-90ef-58950b10598b","Type":"ContainerStarted","Data":"d80bae519a1f707249978c28440e404c63c3f87c82e8dc59edb4c9bae1f4e30e"} Jul 28 16:04:24.904450 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.904194 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:24.904450 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.904218 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" event={"ID":"3616399a-0634-4f02-90ef-58950b10598b","Type":"ContainerStarted","Data":"08fd03211ed8f458aa3ee11e00eb634c38ebe17f482bffe5b1585423bae785ad"} Jul 28 16:04:24.905313 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.905286 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" event={"ID":"f26a3188-b900-4f3a-88d3-2f9a856e48b7","Type":"ContainerStarted","Data":"4c8aac28034b7ae52bb739b2c7bac97bcdd188c291ec0a09ceea0f2dc8c1d770"} Jul 28 16:04:24.906943 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.906922 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ngzg7" event={"ID":"ddb31891-1896-4c34-97c8-6e45e42d349c","Type":"ContainerStarted","Data":"24fefb7e1132ef6097d88db950c147d6f523f49cb217bdb1b2cc27e3080cc9e3"} Jul 28 16:04:24.907052 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.906969 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ngzg7" event={"ID":"ddb31891-1896-4c34-97c8-6e45e42d349c","Type":"ContainerStarted","Data":"a1b832ac4ee8107c524da753c77f262897ad54000ef6f6970d946b5728076a57"} Jul 28 16:04:24.937648 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.937603 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" podStartSLOduration=1.8859412660000001 podStartE2EDuration="4.937586422s" totalImagesPullingTime="3.051645156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:04:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:04:24.936490703 +0000 UTC m=+132.970658030" watchObservedRunningTime="2026-07-28 16:04:24.937586422 +0000 UTC m=+132.971753748" Jul 28 16:04:24.954562 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:24.954502 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ngzg7" podStartSLOduration=131.549122317 podStartE2EDuration="2m12.954481809s" totalImagesPullingTime="1.405359492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:04:24.952967757 +0000 UTC m=+132.987135084" watchObservedRunningTime="2026-07-28 16:04:24.954481809 +0000 UTC m=+132.988649138" Jul 28 16:04:25.016596 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:25.016566 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84945dcd9-hzqsj"] Jul 28 16:04:25.019325 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:04:25.019295 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ca9ca85_d92d_402b_b518_37c0f5d157fc.slice/crio-328ca92000f2eb5577de63b5b0522e927ba5da6a8d0f7a48b0965138ec10e053 WatchSource:0}: Error finding container 328ca92000f2eb5577de63b5b0522e927ba5da6a8d0f7a48b0965138ec10e053: Status 404 returned error can't find the container with id 328ca92000f2eb5577de63b5b0522e927ba5da6a8d0f7a48b0965138ec10e053 Jul 28 16:04:25.094385 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:25.094355 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:25.196482 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:25.196418 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:25.298756 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:25.298727 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:25.913049 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:25.913016 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84945dcd9-hzqsj" event={"ID":"2ca9ca85-d92d-402b-b518-37c0f5d157fc","Type":"ContainerStarted","Data":"328ca92000f2eb5577de63b5b0522e927ba5da6a8d0f7a48b0965138ec10e053"} Jul 28 16:04:25.914594 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:25.914569 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" event={"ID":"f26a3188-b900-4f3a-88d3-2f9a856e48b7","Type":"ContainerStarted","Data":"8c198e0bed59fc506fd830c0bef472ac206266226de84bf93fc9ea9961ee8494"} Jul 28 16:04:25.915151 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:25.915130 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" Jul 28 16:04:25.916282 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:25.916257 2550 patch_prober.go:28] interesting pod/monitoring-plugin-68b7fc9574-xn5jk container/monitoring-plugin namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.133.0.18:9443/health\": dial tcp 10.133.0.18:9443: connect: connection refused" start-of-body= Jul 28 16:04:25.916363 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:25.916294 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" podUID="f26a3188-b900-4f3a-88d3-2f9a856e48b7" containerName="monitoring-plugin" probeResult="failure" output="Get \"https://10.133.0.18:9443/health\": dial tcp 10.133.0.18:9443: connect: connection refused" Jul 28 16:04:25.935547 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:25.935503 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" podStartSLOduration=1.505857274 podStartE2EDuration="2.93549038s" totalImagesPullingTime="1.429633106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:04:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:04:25.934560009 +0000 UTC m=+133.968727335" watchObservedRunningTime="2026-07-28 16:04:25.93549038 +0000 UTC m=+133.969657705" Jul 28 16:04:26.922503 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:26.922472 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-xn5jk" Jul 28 16:04:27.995436 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:27.995384 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:28.099233 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:28.099159 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:28.833585 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:28.833558 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-757d4dcc58-mqzbm" Jul 28 16:04:28.988287 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:28.988228 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:29.089965 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:29.089888 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:30.921226 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:30.921197 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6c878fcd8d-tp2ct" Jul 28 16:04:30.945072 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:30.945043 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84945dcd9-hzqsj" event={"ID":"2ca9ca85-d92d-402b-b518-37c0f5d157fc","Type":"ContainerStarted","Data":"ed74e6d51d18f25d7f5b7c949ec62f9fbe6adf89474170a776ba25f89530a379"} Jul 28 16:04:30.969243 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:30.969191 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84945dcd9-hzqsj" podStartSLOduration=1.460038205 podStartE2EDuration="6.969174147s" totalImagesPullingTime="5.509135942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:04:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:04:30.967904583 +0000 UTC m=+139.002071912" watchObservedRunningTime="2026-07-28 16:04:30.969174147 +0000 UTC m=+139.003341474" Jul 28 16:04:31.410613 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.410577 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84945dcd9-hzqsj"] Jul 28 16:04:31.444038 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.444012 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-777695f965-5vcdd"] Jul 28 16:04:31.447418 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.447386 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.460517 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.460492 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-777695f965-5vcdd"] Jul 28 16:04:31.535161 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.535134 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlhb4\" (UniqueName: \"kubernetes.io/projected/9ab2f35d-465b-43a2-81b6-b605f200e908-kube-api-access-tlhb4\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.535262 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.535166 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ab2f35d-465b-43a2-81b6-b605f200e908-console-oauth-config\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.535262 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.535185 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-trusted-ca-bundle\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.535262 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.535211 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-oauth-serving-cert\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.535262 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.535245 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ab2f35d-465b-43a2-81b6-b605f200e908-console-serving-cert\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.535387 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.535263 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-console-config\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.535387 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.535321 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-service-ca\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.635638 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.635613 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-service-ca\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.635724 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.635652 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlhb4\" (UniqueName: \"kubernetes.io/projected/9ab2f35d-465b-43a2-81b6-b605f200e908-kube-api-access-tlhb4\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.635792 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.635769 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ab2f35d-465b-43a2-81b6-b605f200e908-console-oauth-config\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.635830 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.635817 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-trusted-ca-bundle\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.635863 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.635849 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-oauth-serving-cert\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.635895 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.635869 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ab2f35d-465b-43a2-81b6-b605f200e908-console-serving-cert\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.635942 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.635899 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-console-config\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.636397 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.636367 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-service-ca\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.636540 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.636520 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-console-config\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.636591 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.636557 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-oauth-serving-cert\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.636680 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.636660 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-trusted-ca-bundle\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.638259 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.638239 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ab2f35d-465b-43a2-81b6-b605f200e908-console-serving-cert\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.638356 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.638337 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ab2f35d-465b-43a2-81b6-b605f200e908-console-oauth-config\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.644368 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.644343 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlhb4\" (UniqueName: \"kubernetes.io/projected/9ab2f35d-465b-43a2-81b6-b605f200e908-kube-api-access-tlhb4\") pod \"console-777695f965-5vcdd\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.756723 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.756669 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:31.824704 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.824675 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:31.881309 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.881226 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-777695f965-5vcdd"] Jul 28 16:04:31.883946 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:04:31.883922 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab2f35d_465b_43a2_81b6_b605f200e908.slice/crio-c78dbc9689ab47d1c2474cc624fba597fa5b80d1f97b296030cbe0adea4276f9 WatchSource:0}: Error finding container c78dbc9689ab47d1c2474cc624fba597fa5b80d1f97b296030cbe0adea4276f9: Status 404 returned error can't find the container with id c78dbc9689ab47d1c2474cc624fba597fa5b80d1f97b296030cbe0adea4276f9 Jul 28 16:04:31.950974 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.950939 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777695f965-5vcdd" event={"ID":"9ab2f35d-465b-43a2-81b6-b605f200e908","Type":"ContainerStarted","Data":"43ce1f54958b803bd6b6d1b50e9459eb4b50d9cb2a964466551b6e0c417063c8"} Jul 28 16:04:31.951326 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.950994 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777695f965-5vcdd" event={"ID":"9ab2f35d-465b-43a2-81b6-b605f200e908","Type":"ContainerStarted","Data":"c78dbc9689ab47d1c2474cc624fba597fa5b80d1f97b296030cbe0adea4276f9"} Jul 28 16:04:31.977667 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:31.977624 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-777695f965-5vcdd" podStartSLOduration=0.977608427 podStartE2EDuration="977.608427ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:04:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:04:31.974681067 +0000 UTC m=+140.008848394" watchObservedRunningTime="2026-07-28 16:04:31.977608427 +0000 UTC m=+140.011775752" Jul 28 16:04:32.413959 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:32.413929 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:34.882593 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:34.882562 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:35.131397 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:35.131363 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:41.757542 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:41.757503 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:41.758051 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:41.757565 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:41.762506 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:41.762472 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:41.998104 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:41.998076 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:04:42.165877 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:42.165843 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:42.267073 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:42.267043 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:42.332892 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:42.332867 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:42.991586 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:42.991547 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:43.052118 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:43.052083 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:44.228130 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:44.228090 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:44.328560 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:44.328523 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:44.401610 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:44.401572 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:45.662086 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:45.662046 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:45.733980 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:45.733953 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:51.890224 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:51.890177 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:51.948222 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:51.948189 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:57.057339 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:57.057309 2550 generic.go:356] "Generic (PLEG): container finished" podID="8f454402-1f88-4231-a0f5-5bc2f58d0a01" containerID="8d1555d8359edf3201995e8d5770aa1f0cd9a3f888cc813a610aadee19d13118" exitCode=0 Jul 28 16:04:57.057653 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:57.057382 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-lxlwg" event={"ID":"8f454402-1f88-4231-a0f5-5bc2f58d0a01","Type":"ContainerDied","Data":"8d1555d8359edf3201995e8d5770aa1f0cd9a3f888cc813a610aadee19d13118"} Jul 28 16:04:57.057719 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:57.057702 2550 scope.go:117] "RemoveContainer" containerID="8d1555d8359edf3201995e8d5770aa1f0cd9a3f888cc813a610aadee19d13118" Jul 28 16:04:57.973545 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:57.973498 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84945dcd9-hzqsj" podUID="2ca9ca85-d92d-402b-b518-37c0f5d157fc" containerName="console" containerID="cri-o://ed74e6d51d18f25d7f5b7c949ec62f9fbe6adf89474170a776ba25f89530a379" gracePeriod=15 Jul 28 16:04:57.986363 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:57.986341 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:58.066960 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.066931 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-lxlwg" event={"ID":"8f454402-1f88-4231-a0f5-5bc2f58d0a01","Type":"ContainerStarted","Data":"3f3ea1b3723b6a0034bebb28a2ca810a993cb1604479d7f0c414fc8c3816b683"} Jul 28 16:04:58.079269 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.079242 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:58.209653 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.209630 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:58.317852 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.317828 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-oauth-config\") pod \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " Jul 28 16:04:58.317956 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.317869 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z74z5\" (UniqueName: \"kubernetes.io/projected/2ca9ca85-d92d-402b-b518-37c0f5d157fc-kube-api-access-z74z5\") pod \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " Jul 28 16:04:58.317956 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.317889 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-serving-cert\") pod \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " Jul 28 16:04:58.317956 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.317911 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-service-ca\") pod \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " Jul 28 16:04:58.317956 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.317946 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-trusted-ca-bundle\") pod \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " Jul 28 16:04:58.318103 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.318018 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-config\") pod \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " Jul 28 16:04:58.318103 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.318068 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-oauth-serving-cert\") pod \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\" (UID: \"2ca9ca85-d92d-402b-b518-37c0f5d157fc\") " Jul 28 16:04:58.318386 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.318347 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-service-ca" (OuterVolumeSpecName: "service-ca") pod "2ca9ca85-d92d-402b-b518-37c0f5d157fc" (UID: "2ca9ca85-d92d-402b-b518-37c0f5d157fc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:04:58.318529 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.318385 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2ca9ca85-d92d-402b-b518-37c0f5d157fc" (UID: "2ca9ca85-d92d-402b-b518-37c0f5d157fc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:04:58.318620 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.318520 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-config" (OuterVolumeSpecName: "console-config") pod "2ca9ca85-d92d-402b-b518-37c0f5d157fc" (UID: "2ca9ca85-d92d-402b-b518-37c0f5d157fc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:04:58.318685 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.318612 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2ca9ca85-d92d-402b-b518-37c0f5d157fc" (UID: "2ca9ca85-d92d-402b-b518-37c0f5d157fc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:04:58.320067 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.320040 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2ca9ca85-d92d-402b-b518-37c0f5d157fc" (UID: "2ca9ca85-d92d-402b-b518-37c0f5d157fc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:04:58.320164 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.320099 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2ca9ca85-d92d-402b-b518-37c0f5d157fc" (UID: "2ca9ca85-d92d-402b-b518-37c0f5d157fc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:04:58.320164 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.320110 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ca9ca85-d92d-402b-b518-37c0f5d157fc-kube-api-access-z74z5" (OuterVolumeSpecName: "kube-api-access-z74z5") pod "2ca9ca85-d92d-402b-b518-37c0f5d157fc" (UID: "2ca9ca85-d92d-402b-b518-37c0f5d157fc"). InnerVolumeSpecName "kube-api-access-z74z5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:04:58.418961 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.418936 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-oauth-config\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:04:58.418961 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.418957 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z74z5\" (UniqueName: \"kubernetes.io/projected/2ca9ca85-d92d-402b-b518-37c0f5d157fc-kube-api-access-z74z5\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:04:58.419084 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.418967 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-serving-cert\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:04:58.419084 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.418976 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-service-ca\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:04:58.419084 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.418984 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-trusted-ca-bundle\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:04:58.419084 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.418992 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-console-config\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:04:58.419084 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.419001 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ca9ca85-d92d-402b-b518-37c0f5d157fc-oauth-serving-cert\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:04:58.854641 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.854609 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:58.930790 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:58.930755 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:04:59.072500 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:59.072473 2550 generic.go:356] "Generic (PLEG): container finished" podID="2ca9ca85-d92d-402b-b518-37c0f5d157fc" containerID="ed74e6d51d18f25d7f5b7c949ec62f9fbe6adf89474170a776ba25f89530a379" exitCode=0 Jul 28 16:04:59.072907 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:59.072549 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84945dcd9-hzqsj" Jul 28 16:04:59.072907 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:59.072540 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84945dcd9-hzqsj" event={"ID":"2ca9ca85-d92d-402b-b518-37c0f5d157fc","Type":"ContainerDied","Data":"ed74e6d51d18f25d7f5b7c949ec62f9fbe6adf89474170a776ba25f89530a379"} Jul 28 16:04:59.072907 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:59.072693 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84945dcd9-hzqsj" event={"ID":"2ca9ca85-d92d-402b-b518-37c0f5d157fc","Type":"ContainerDied","Data":"328ca92000f2eb5577de63b5b0522e927ba5da6a8d0f7a48b0965138ec10e053"} Jul 28 16:04:59.072907 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:59.072730 2550 scope.go:117] "RemoveContainer" containerID="ed74e6d51d18f25d7f5b7c949ec62f9fbe6adf89474170a776ba25f89530a379" Jul 28 16:04:59.081852 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:59.081808 2550 scope.go:117] "RemoveContainer" containerID="ed74e6d51d18f25d7f5b7c949ec62f9fbe6adf89474170a776ba25f89530a379" Jul 28 16:04:59.082172 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:04:59.082148 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed74e6d51d18f25d7f5b7c949ec62f9fbe6adf89474170a776ba25f89530a379\": container with ID starting with ed74e6d51d18f25d7f5b7c949ec62f9fbe6adf89474170a776ba25f89530a379 not found: ID does not exist" containerID="ed74e6d51d18f25d7f5b7c949ec62f9fbe6adf89474170a776ba25f89530a379" Jul 28 16:04:59.082256 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:59.082182 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed74e6d51d18f25d7f5b7c949ec62f9fbe6adf89474170a776ba25f89530a379"} err="failed to get container status \"ed74e6d51d18f25d7f5b7c949ec62f9fbe6adf89474170a776ba25f89530a379\": rpc error: code = NotFound desc = could not find container \"ed74e6d51d18f25d7f5b7c949ec62f9fbe6adf89474170a776ba25f89530a379\": container with ID starting with ed74e6d51d18f25d7f5b7c949ec62f9fbe6adf89474170a776ba25f89530a379 not found: ID does not exist" Jul 28 16:04:59.091117 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:59.091087 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84945dcd9-hzqsj"] Jul 28 16:04:59.097256 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:04:59.097223 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84945dcd9-hzqsj"] Jul 28 16:05:00.441083 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:00.441032 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ca9ca85-d92d-402b-b518-37c0f5d157fc" path="/var/lib/kubelet/pods/2ca9ca85-d92d-402b-b518-37c0f5d157fc/volumes" Jul 28 16:05:34.718607 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:34.718562 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:05:34.782791 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:34.782762 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:05:36.452175 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:36.452137 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:05:40.982795 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:40.982759 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:05:40.983284 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:40.983045 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ca9ca85-d92d-402b-b518-37c0f5d157fc" containerName="console" Jul 28 16:05:40.983284 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:40.983056 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ca9ca85-d92d-402b-b518-37c0f5d157fc" containerName="console" Jul 28 16:05:40.983284 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:40.983117 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ca9ca85-d92d-402b-b518-37c0f5d157fc" containerName="console" Jul 28 16:05:40.990212 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:40.990187 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:40.994682 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:40.994659 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 16:05:40.994796 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:40.994747 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 16:05:41.000453 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.000434 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 16:05:41.003035 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.003021 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 16:05:41.006017 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.006001 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 16:05:41.006455 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.006441 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 16:05:41.007285 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.007270 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 16:05:41.020765 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.020738 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 16:05:41.023309 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.023290 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:05:41.027018 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.026994 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qm5vd\"" Jul 28 16:05:41.032333 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.032312 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 16:05:41.133055 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.133033 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.133148 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.133065 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-config-out\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.133148 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.133085 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.133148 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.133102 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.133148 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.133131 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.133273 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.133151 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.133273 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.133199 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.133273 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.133222 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.133273 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.133245 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.133396 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.133299 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-web-config\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.133396 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.133317 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.133396 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.133335 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-config-volume\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.133396 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.133387 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbtrz\" (UniqueName: \"kubernetes.io/projected/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-kube-api-access-lbtrz\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.233929 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.233870 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-config-volume\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.233929 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.233907 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbtrz\" (UniqueName: \"kubernetes.io/projected/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-kube-api-access-lbtrz\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.234088 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.233929 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.234088 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.233954 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-config-out\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.234088 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.233973 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.234088 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.233993 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.234088 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.234023 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.234088 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.234058 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.234488 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.234092 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.234488 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.234117 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.234488 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.234145 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.234488 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.234181 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-web-config\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.234488 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.234206 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.234762 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.234610 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.235396 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.235030 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.237567 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.237263 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-config-out\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.237567 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.237336 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.237567 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.237464 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.237730 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.237594 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.237909 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.237883 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-config-volume\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.237982 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.237942 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.237982 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.237964 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.238098 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.238018 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.238280 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.238257 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.239627 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.239608 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-web-config\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.245602 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.245578 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbtrz\" (UniqueName: \"kubernetes.io/projected/38a6f237-40ab-4a17-98b5-b7d66c8e81dd-kube-api-access-lbtrz\") pod \"alertmanager-main-0\" (UID: \"38a6f237-40ab-4a17-98b5-b7d66c8e81dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.299456 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.299436 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:05:41.437563 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:41.437532 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:05:41.441205 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:05:41.441175 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a6f237_40ab_4a17_98b5_b7d66c8e81dd.slice/crio-58f67596935a10266bca0aa0668f8b31976699f805f9dcf9549987ed5937a47f WatchSource:0}: Error finding container 58f67596935a10266bca0aa0668f8b31976699f805f9dcf9549987ed5937a47f: Status 404 returned error can't find the container with id 58f67596935a10266bca0aa0668f8b31976699f805f9dcf9549987ed5937a47f Jul 28 16:05:42.256688 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:42.256647 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38a6f237-40ab-4a17-98b5-b7d66c8e81dd","Type":"ContainerStarted","Data":"58f67596935a10266bca0aa0668f8b31976699f805f9dcf9549987ed5937a47f"} Jul 28 16:05:44.266501 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:44.266462 2550 generic.go:356] "Generic (PLEG): container finished" podID="38a6f237-40ab-4a17-98b5-b7d66c8e81dd" containerID="a89379aa7db9db6aa1c3a9bb9d996f937f210ec262bbc19b0cac002ad506e404" exitCode=0 Jul 28 16:05:44.266868 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:44.266554 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38a6f237-40ab-4a17-98b5-b7d66c8e81dd","Type":"ContainerDied","Data":"a89379aa7db9db6aa1c3a9bb9d996f937f210ec262bbc19b0cac002ad506e404"} Jul 28 16:05:46.570706 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:46.570664 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:05:46.626737 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:46.626700 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:05:47.904427 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:47.904370 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c4f4b4fd9-4zr2p"] Jul 28 16:05:47.908255 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:47.908226 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:47.926150 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:47.926119 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c4f4b4fd9-4zr2p"] Jul 28 16:05:47.981810 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:47.981785 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-oauth-serving-cert\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:47.981938 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:47.981826 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvh77\" (UniqueName: \"kubernetes.io/projected/329c6e87-1101-41e3-8be1-e4d8eadd71cf-kube-api-access-mvh77\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:47.981938 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:47.981848 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-serving-cert\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:47.981938 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:47.981886 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-trusted-ca-bundle\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:47.981938 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:47.981920 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-oauth-config\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:47.981938 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:47.981936 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-config\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:47.982123 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:47.981952 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-service-ca\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.082531 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.082499 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-oauth-config\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.082667 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.082542 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-config\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.082667 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.082575 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-service-ca\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.082667 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.082624 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-oauth-serving-cert\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.082807 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.082671 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvh77\" (UniqueName: \"kubernetes.io/projected/329c6e87-1101-41e3-8be1-e4d8eadd71cf-kube-api-access-mvh77\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.082807 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.082796 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-serving-cert\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.082911 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.082850 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-trusted-ca-bundle\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.083365 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.083340 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-service-ca\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.083489 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.083340 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-config\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.083710 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.083658 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-oauth-serving-cert\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.083831 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.083739 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-trusted-ca-bundle\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.085388 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.085362 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-oauth-config\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.085859 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.085834 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-serving-cert\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.092431 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.092378 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvh77\" (UniqueName: \"kubernetes.io/projected/329c6e87-1101-41e3-8be1-e4d8eadd71cf-kube-api-access-mvh77\") pod \"console-5c4f4b4fd9-4zr2p\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.225888 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.225825 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:48.359240 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:48.359173 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c4f4b4fd9-4zr2p"] Jul 28 16:05:48.362643 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:05:48.362606 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod329c6e87_1101_41e3_8be1_e4d8eadd71cf.slice/crio-d0b7397d5759556aee0441add05f79fb830e14a24673e0ecf2b74281ec3559da WatchSource:0}: Error finding container d0b7397d5759556aee0441add05f79fb830e14a24673e0ecf2b74281ec3559da: Status 404 returned error can't find the container with id d0b7397d5759556aee0441add05f79fb830e14a24673e0ecf2b74281ec3559da Jul 28 16:05:49.290186 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:49.290153 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38a6f237-40ab-4a17-98b5-b7d66c8e81dd","Type":"ContainerStarted","Data":"f74b838946242ffccafc639f0983bc39a188af269bf6af99fbaf6f042442c0ac"} Jul 28 16:05:49.290537 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:49.290236 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38a6f237-40ab-4a17-98b5-b7d66c8e81dd","Type":"ContainerStarted","Data":"0c7fbeead99b3b301e0a577d91d072f6ed88c148e1444acccab9b33c276397da"} Jul 28 16:05:49.290537 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:49.290286 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38a6f237-40ab-4a17-98b5-b7d66c8e81dd","Type":"ContainerStarted","Data":"7eb864b1e1021b17ab86639ba32d69e1ec148558ebe1cf209a1aee6bd807c7d0"} Jul 28 16:05:49.290537 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:49.290329 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38a6f237-40ab-4a17-98b5-b7d66c8e81dd","Type":"ContainerStarted","Data":"e91f3f81fe1e659539234658f88081b04ead32676b2ddfd83e50b101a17e3f32"} Jul 28 16:05:49.290537 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:49.290373 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38a6f237-40ab-4a17-98b5-b7d66c8e81dd","Type":"ContainerStarted","Data":"dd062727edc8ec3ce5efaef15bd3c19560c3d6e6077790ba42fdf5bed752d2e5"} Jul 28 16:05:49.290537 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:49.290433 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38a6f237-40ab-4a17-98b5-b7d66c8e81dd","Type":"ContainerStarted","Data":"7bfad84df3c532c112d1b35738ba9967885c61636b05ac5cc70623881022b715"} Jul 28 16:05:49.291822 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:49.291800 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4f4b4fd9-4zr2p" event={"ID":"329c6e87-1101-41e3-8be1-e4d8eadd71cf","Type":"ContainerStarted","Data":"666d145a1e1ca668d7ce4d7c5f5ef50c76ade296e969a31ed1c3192fee2adece"} Jul 28 16:05:49.291911 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:49.291844 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4f4b4fd9-4zr2p" event={"ID":"329c6e87-1101-41e3-8be1-e4d8eadd71cf","Type":"ContainerStarted","Data":"d0b7397d5759556aee0441add05f79fb830e14a24673e0ecf2b74281ec3559da"} Jul 28 16:05:49.320979 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:49.320939 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.840996807 podStartE2EDuration="9.320923988s" totalImagesPullingTime="6.46443923s" totalInitContainerRuntime="15.487951ms" isStatefulPod=false podCreationTimestamp="2026-07-28 16:05:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:05:49.319522698 +0000 UTC m=+217.353690024" watchObservedRunningTime="2026-07-28 16:05:49.320923988 +0000 UTC m=+217.355091314" Jul 28 16:05:49.342786 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:49.342736 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c4f4b4fd9-4zr2p" podStartSLOduration=2.34271794 podStartE2EDuration="2.34271794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:05:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:05:49.341321098 +0000 UTC m=+217.375488427" watchObservedRunningTime="2026-07-28 16:05:49.34271794 +0000 UTC m=+217.376885267" Jul 28 16:05:51.014377 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:51.014338 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:05:51.080804 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:51.080776 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:05:54.245040 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:54.245001 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:05:57.995063 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:57.995025 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:05:58.095733 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:58.095699 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:05:58.226463 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:58.226428 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:58.226606 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:58.226479 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:58.231273 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:58.231248 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:58.276168 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:58.276100 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:05:58.332543 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:58.332520 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:05:58.385818 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:58.385788 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-777695f965-5vcdd"] Jul 28 16:05:58.506741 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:58.506710 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:05:58.562974 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:58.562950 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:05:58.620876 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:05:58.620849 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:06:23.410445 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.410361 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-777695f965-5vcdd" podUID="9ab2f35d-465b-43a2-81b6-b605f200e908" containerName="console" containerID="cri-o://43ce1f54958b803bd6b6d1b50e9459eb4b50d9cb2a964466551b6e0c417063c8" gracePeriod=15 Jul 28 16:06:23.647256 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.647233 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:06:23.738143 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.738087 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-console-config\") pod \"9ab2f35d-465b-43a2-81b6-b605f200e908\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " Jul 28 16:06:23.738143 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.738119 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-service-ca\") pod \"9ab2f35d-465b-43a2-81b6-b605f200e908\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " Jul 28 16:06:23.738306 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.738154 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlhb4\" (UniqueName: \"kubernetes.io/projected/9ab2f35d-465b-43a2-81b6-b605f200e908-kube-api-access-tlhb4\") pod \"9ab2f35d-465b-43a2-81b6-b605f200e908\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " Jul 28 16:06:23.738306 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.738281 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ab2f35d-465b-43a2-81b6-b605f200e908-console-oauth-config\") pod \"9ab2f35d-465b-43a2-81b6-b605f200e908\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " Jul 28 16:06:23.738435 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.738340 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-oauth-serving-cert\") pod \"9ab2f35d-465b-43a2-81b6-b605f200e908\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " Jul 28 16:06:23.738435 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.738394 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-trusted-ca-bundle\") pod \"9ab2f35d-465b-43a2-81b6-b605f200e908\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " Jul 28 16:06:23.738552 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.738447 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ab2f35d-465b-43a2-81b6-b605f200e908-console-serving-cert\") pod \"9ab2f35d-465b-43a2-81b6-b605f200e908\" (UID: \"9ab2f35d-465b-43a2-81b6-b605f200e908\") " Jul 28 16:06:23.738552 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.738491 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-service-ca" (OuterVolumeSpecName: "service-ca") pod "9ab2f35d-465b-43a2-81b6-b605f200e908" (UID: "9ab2f35d-465b-43a2-81b6-b605f200e908"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:06:23.738552 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.738502 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-console-config" (OuterVolumeSpecName: "console-config") pod "9ab2f35d-465b-43a2-81b6-b605f200e908" (UID: "9ab2f35d-465b-43a2-81b6-b605f200e908"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:06:23.738788 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.738771 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-service-ca\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:06:23.738847 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.738800 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-console-config\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:06:23.738847 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.738766 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9ab2f35d-465b-43a2-81b6-b605f200e908" (UID: "9ab2f35d-465b-43a2-81b6-b605f200e908"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:06:23.738946 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.738919 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9ab2f35d-465b-43a2-81b6-b605f200e908" (UID: "9ab2f35d-465b-43a2-81b6-b605f200e908"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:06:23.740279 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.740253 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ab2f35d-465b-43a2-81b6-b605f200e908-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9ab2f35d-465b-43a2-81b6-b605f200e908" (UID: "9ab2f35d-465b-43a2-81b6-b605f200e908"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:06:23.740376 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.740318 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ab2f35d-465b-43a2-81b6-b605f200e908-kube-api-access-tlhb4" (OuterVolumeSpecName: "kube-api-access-tlhb4") pod "9ab2f35d-465b-43a2-81b6-b605f200e908" (UID: "9ab2f35d-465b-43a2-81b6-b605f200e908"). InnerVolumeSpecName "kube-api-access-tlhb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:06:23.740478 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.740392 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ab2f35d-465b-43a2-81b6-b605f200e908-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9ab2f35d-465b-43a2-81b6-b605f200e908" (UID: "9ab2f35d-465b-43a2-81b6-b605f200e908"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:06:23.840004 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.839984 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ab2f35d-465b-43a2-81b6-b605f200e908-console-serving-cert\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:06:23.840004 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.840005 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlhb4\" (UniqueName: \"kubernetes.io/projected/9ab2f35d-465b-43a2-81b6-b605f200e908-kube-api-access-tlhb4\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:06:23.840124 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.840014 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ab2f35d-465b-43a2-81b6-b605f200e908-console-oauth-config\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:06:23.840124 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.840023 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-oauth-serving-cert\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:06:23.840124 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:23.840033 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ab2f35d-465b-43a2-81b6-b605f200e908-trusted-ca-bundle\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:06:24.441361 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:24.441322 2550 generic.go:356] "Generic (PLEG): container finished" podID="9ab2f35d-465b-43a2-81b6-b605f200e908" containerID="43ce1f54958b803bd6b6d1b50e9459eb4b50d9cb2a964466551b6e0c417063c8" exitCode=0 Jul 28 16:06:24.441777 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:24.441442 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-777695f965-5vcdd" Jul 28 16:06:24.441960 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:24.441933 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777695f965-5vcdd" event={"ID":"9ab2f35d-465b-43a2-81b6-b605f200e908","Type":"ContainerDied","Data":"43ce1f54958b803bd6b6d1b50e9459eb4b50d9cb2a964466551b6e0c417063c8"} Jul 28 16:06:24.441996 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:24.441982 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-777695f965-5vcdd" event={"ID":"9ab2f35d-465b-43a2-81b6-b605f200e908","Type":"ContainerDied","Data":"c78dbc9689ab47d1c2474cc624fba597fa5b80d1f97b296030cbe0adea4276f9"} Jul 28 16:06:24.442035 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:24.442005 2550 scope.go:117] "RemoveContainer" containerID="43ce1f54958b803bd6b6d1b50e9459eb4b50d9cb2a964466551b6e0c417063c8" Jul 28 16:06:24.451028 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:24.451009 2550 scope.go:117] "RemoveContainer" containerID="43ce1f54958b803bd6b6d1b50e9459eb4b50d9cb2a964466551b6e0c417063c8" Jul 28 16:06:24.451325 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:06:24.451306 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43ce1f54958b803bd6b6d1b50e9459eb4b50d9cb2a964466551b6e0c417063c8\": container with ID starting with 43ce1f54958b803bd6b6d1b50e9459eb4b50d9cb2a964466551b6e0c417063c8 not found: ID does not exist" containerID="43ce1f54958b803bd6b6d1b50e9459eb4b50d9cb2a964466551b6e0c417063c8" Jul 28 16:06:24.451387 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:24.451332 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43ce1f54958b803bd6b6d1b50e9459eb4b50d9cb2a964466551b6e0c417063c8"} err="failed to get container status \"43ce1f54958b803bd6b6d1b50e9459eb4b50d9cb2a964466551b6e0c417063c8\": rpc error: code = NotFound desc = could not find container \"43ce1f54958b803bd6b6d1b50e9459eb4b50d9cb2a964466551b6e0c417063c8\": container with ID starting with 43ce1f54958b803bd6b6d1b50e9459eb4b50d9cb2a964466551b6e0c417063c8 not found: ID does not exist" Jul 28 16:06:24.470715 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:24.470662 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-777695f965-5vcdd"] Jul 28 16:06:24.472618 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:24.472595 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-777695f965-5vcdd"] Jul 28 16:06:26.440565 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:26.440526 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ab2f35d-465b-43a2-81b6-b605f200e908" path="/var/lib/kubelet/pods/9ab2f35d-465b-43a2-81b6-b605f200e908/volumes" Jul 28 16:06:28.044133 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:28.044093 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:06:28.099869 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:28.099838 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:06:43.435624 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:43.435525 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:06:43.531445 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:43.531395 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:06:43.885670 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:43.885637 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:06:43.959968 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:43.959941 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:06:44.029629 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:44.029596 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:06:44.903005 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:44.900214 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w"] Jul 28 16:06:44.903005 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:44.901018 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ab2f35d-465b-43a2-81b6-b605f200e908" containerName="console" Jul 28 16:06:44.903005 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:44.901042 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ab2f35d-465b-43a2-81b6-b605f200e908" containerName="console" Jul 28 16:06:44.903005 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:44.901190 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ab2f35d-465b-43a2-81b6-b605f200e908" containerName="console" Jul 28 16:06:44.905676 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:44.905634 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" Jul 28 16:06:44.908291 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:44.908257 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-twdwk\"" Jul 28 16:06:44.908291 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:44.908285 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 28 16:06:44.908485 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:44.908289 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 28 16:06:44.911208 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:44.911184 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w"] Jul 28 16:06:44.984003 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:44.983978 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w\" (UID: \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" Jul 28 16:06:44.984123 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:44.984008 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnbq4\" (UniqueName: \"kubernetes.io/projected/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-kube-api-access-nnbq4\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w\" (UID: \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" Jul 28 16:06:44.984123 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:44.984045 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w\" (UID: \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" Jul 28 16:06:45.085009 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:45.084980 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w\" (UID: \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" Jul 28 16:06:45.085147 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:45.085023 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnbq4\" (UniqueName: \"kubernetes.io/projected/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-kube-api-access-nnbq4\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w\" (UID: \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" Jul 28 16:06:45.085147 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:45.085085 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w\" (UID: \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" Jul 28 16:06:45.085382 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:45.085355 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w\" (UID: \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" Jul 28 16:06:45.085656 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:45.085456 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w\" (UID: \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" Jul 28 16:06:45.096248 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:45.096214 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnbq4\" (UniqueName: \"kubernetes.io/projected/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-kube-api-access-nnbq4\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w\" (UID: \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" Jul 28 16:06:45.217592 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:45.217530 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" Jul 28 16:06:45.351628 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:45.351606 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w"] Jul 28 16:06:45.353765 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:06:45.353732 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0e95e3_c644_4df5_bf84_c12c21cc3a38.slice/crio-ca8dfe3bf8572f499aae1d0521f0a77f793d6d6662eaf842a997bb21cfd95e0f WatchSource:0}: Error finding container ca8dfe3bf8572f499aae1d0521f0a77f793d6d6662eaf842a997bb21cfd95e0f: Status 404 returned error can't find the container with id ca8dfe3bf8572f499aae1d0521f0a77f793d6d6662eaf842a997bb21cfd95e0f Jul 28 16:06:45.522595 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:45.522526 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" event={"ID":"5e0e95e3-c644-4df5-bf84-c12c21cc3a38","Type":"ContainerStarted","Data":"ca8dfe3bf8572f499aae1d0521f0a77f793d6d6662eaf842a997bb21cfd95e0f"} Jul 28 16:06:51.551238 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:51.551206 2550 generic.go:356] "Generic (PLEG): container finished" podID="5e0e95e3-c644-4df5-bf84-c12c21cc3a38" containerID="a19896c624691a10d23541825d449176f546234862ec012a74e487006055ada4" exitCode=0 Jul 28 16:06:51.551651 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:51.551297 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" event={"ID":"5e0e95e3-c644-4df5-bf84-c12c21cc3a38","Type":"ContainerDied","Data":"a19896c624691a10d23541825d449176f546234862ec012a74e487006055ada4"} Jul 28 16:06:54.567187 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:54.567148 2550 generic.go:356] "Generic (PLEG): container finished" podID="5e0e95e3-c644-4df5-bf84-c12c21cc3a38" containerID="d80a1c5a1a8e3b677f8b6a89084dc34a4ddeabaa7d0b0e64b4d1a2ef659dd7fa" exitCode=0 Jul 28 16:06:54.567640 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:06:54.567244 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" event={"ID":"5e0e95e3-c644-4df5-bf84-c12c21cc3a38","Type":"ContainerDied","Data":"d80a1c5a1a8e3b677f8b6a89084dc34a4ddeabaa7d0b0e64b4d1a2ef659dd7fa"} Jul 28 16:07:01.604648 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:01.604613 2550 generic.go:356] "Generic (PLEG): container finished" podID="5e0e95e3-c644-4df5-bf84-c12c21cc3a38" containerID="ef20a55029bd87efac95130d35a17955ebedffbf6156b6208cc0690822553958" exitCode=0 Jul 28 16:07:01.605018 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:01.604696 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" event={"ID":"5e0e95e3-c644-4df5-bf84-c12c21cc3a38","Type":"ContainerDied","Data":"ef20a55029bd87efac95130d35a17955ebedffbf6156b6208cc0690822553958"} Jul 28 16:07:02.726766 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:02.726742 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" Jul 28 16:07:02.827918 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:02.827895 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-bundle\") pod \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\" (UID: \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\") " Jul 28 16:07:02.828061 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:02.827964 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-util\") pod \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\" (UID: \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\") " Jul 28 16:07:02.828061 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:02.827999 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnbq4\" (UniqueName: \"kubernetes.io/projected/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-kube-api-access-nnbq4\") pod \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\" (UID: \"5e0e95e3-c644-4df5-bf84-c12c21cc3a38\") " Jul 28 16:07:02.828501 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:02.828477 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-bundle" (OuterVolumeSpecName: "bundle") pod "5e0e95e3-c644-4df5-bf84-c12c21cc3a38" (UID: "5e0e95e3-c644-4df5-bf84-c12c21cc3a38"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:07:02.829959 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:02.829941 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-kube-api-access-nnbq4" (OuterVolumeSpecName: "kube-api-access-nnbq4") pod "5e0e95e3-c644-4df5-bf84-c12c21cc3a38" (UID: "5e0e95e3-c644-4df5-bf84-c12c21cc3a38"). InnerVolumeSpecName "kube-api-access-nnbq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:07:02.832482 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:02.832458 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-util" (OuterVolumeSpecName: "util") pod "5e0e95e3-c644-4df5-bf84-c12c21cc3a38" (UID: "5e0e95e3-c644-4df5-bf84-c12c21cc3a38"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:07:02.929475 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:02.929390 2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-bundle\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:07:02.929475 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:02.929438 2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-util\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:07:02.929475 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:02.929451 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnbq4\" (UniqueName: \"kubernetes.io/projected/5e0e95e3-c644-4df5-bf84-c12c21cc3a38-kube-api-access-nnbq4\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:07:03.614641 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:03.614605 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" event={"ID":"5e0e95e3-c644-4df5-bf84-c12c21cc3a38","Type":"ContainerDied","Data":"ca8dfe3bf8572f499aae1d0521f0a77f793d6d6662eaf842a997bb21cfd95e0f"} Jul 28 16:07:03.614821 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:03.614641 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b549k2w" Jul 28 16:07:03.614821 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:03.614652 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca8dfe3bf8572f499aae1d0521f0a77f793d6d6662eaf842a997bb21cfd95e0f" Jul 28 16:07:05.946797 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:05.946756 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:06.050382 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:06.050350 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:07.136791 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:07.136757 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:12.341050 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:12.341025 2550 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 16:07:17.444883 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:17.444842 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:17.754804 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:17.754726 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:17.858918 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:17.858889 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:18.298084 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.298051 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:18.492113 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.492088 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-46x6s"] Jul 28 16:07:18.492523 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.492506 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e0e95e3-c644-4df5-bf84-c12c21cc3a38" containerName="extract" Jul 28 16:07:18.492582 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.492525 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e0e95e3-c644-4df5-bf84-c12c21cc3a38" containerName="extract" Jul 28 16:07:18.492582 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.492539 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e0e95e3-c644-4df5-bf84-c12c21cc3a38" containerName="util" Jul 28 16:07:18.492582 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.492544 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e0e95e3-c644-4df5-bf84-c12c21cc3a38" containerName="util" Jul 28 16:07:18.492582 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.492551 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e0e95e3-c644-4df5-bf84-c12c21cc3a38" containerName="pull" Jul 28 16:07:18.492582 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.492556 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e0e95e3-c644-4df5-bf84-c12c21cc3a38" containerName="pull" Jul 28 16:07:18.492741 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.492608 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e0e95e3-c644-4df5-bf84-c12c21cc3a38" containerName="extract" Jul 28 16:07:18.494524 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.494500 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" Jul 28 16:07:18.497106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.497082 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 28 16:07:18.497218 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.497116 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 28 16:07:18.497218 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.497138 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Jul 28 16:07:18.497218 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.497084 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 28 16:07:18.497218 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.497082 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 28 16:07:18.497752 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.497738 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-gk8nr\"" Jul 28 16:07:18.504549 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.504528 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-46x6s"] Jul 28 16:07:18.639306 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.639282 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/8494791c-c787-4232-9dad-e6d1afea3823-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-46x6s\" (UID: \"8494791c-c787-4232-9dad-e6d1afea3823\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" Jul 28 16:07:18.639424 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.639316 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5j9m\" (UniqueName: \"kubernetes.io/projected/8494791c-c787-4232-9dad-e6d1afea3823-kube-api-access-j5j9m\") pod \"keda-metrics-apiserver-55557df8c-46x6s\" (UID: \"8494791c-c787-4232-9dad-e6d1afea3823\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" Jul 28 16:07:18.639424 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.639335 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8494791c-c787-4232-9dad-e6d1afea3823-certificates\") pod \"keda-metrics-apiserver-55557df8c-46x6s\" (UID: \"8494791c-c787-4232-9dad-e6d1afea3823\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" Jul 28 16:07:18.740032 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.740002 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/8494791c-c787-4232-9dad-e6d1afea3823-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-46x6s\" (UID: \"8494791c-c787-4232-9dad-e6d1afea3823\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" Jul 28 16:07:18.740138 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.740039 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5j9m\" (UniqueName: \"kubernetes.io/projected/8494791c-c787-4232-9dad-e6d1afea3823-kube-api-access-j5j9m\") pod \"keda-metrics-apiserver-55557df8c-46x6s\" (UID: \"8494791c-c787-4232-9dad-e6d1afea3823\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" Jul 28 16:07:18.740138 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.740059 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8494791c-c787-4232-9dad-e6d1afea3823-certificates\") pod \"keda-metrics-apiserver-55557df8c-46x6s\" (UID: \"8494791c-c787-4232-9dad-e6d1afea3823\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" Jul 28 16:07:18.740213 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:07:18.740161 2550 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found Jul 28 16:07:18.740213 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:07:18.740176 2550 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-55557df8c-46x6s: secret "keda-metrics-apiserver-certs" not found Jul 28 16:07:18.740302 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:07:18.740231 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8494791c-c787-4232-9dad-e6d1afea3823-certificates podName:8494791c-c787-4232-9dad-e6d1afea3823 nodeName:}" failed. No retries permitted until 2026-07-28 16:07:19.240216259 +0000 UTC m=+307.274383564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/8494791c-c787-4232-9dad-e6d1afea3823-certificates") pod "keda-metrics-apiserver-55557df8c-46x6s" (UID: "8494791c-c787-4232-9dad-e6d1afea3823") : secret "keda-metrics-apiserver-certs" not found Jul 28 16:07:18.740364 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.740340 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/8494791c-c787-4232-9dad-e6d1afea3823-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-46x6s\" (UID: \"8494791c-c787-4232-9dad-e6d1afea3823\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" Jul 28 16:07:18.748744 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.748721 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5j9m\" (UniqueName: \"kubernetes.io/projected/8494791c-c787-4232-9dad-e6d1afea3823-kube-api-access-j5j9m\") pod \"keda-metrics-apiserver-55557df8c-46x6s\" (UID: \"8494791c-c787-4232-9dad-e6d1afea3823\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" Jul 28 16:07:18.809923 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.809902 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:18.813935 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.813915 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-2pcbn"] Jul 28 16:07:18.816067 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.816044 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-2pcbn" Jul 28 16:07:18.818153 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.818139 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Jul 28 16:07:18.825776 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.825753 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-2pcbn"] Jul 28 16:07:18.905909 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.905847 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:18.941034 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.941010 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfl8d\" (UniqueName: \"kubernetes.io/projected/3fa39b87-81d8-4aeb-adef-206ac15312a2-kube-api-access-kfl8d\") pod \"keda-admission-56f9798bb4-2pcbn\" (UID: \"3fa39b87-81d8-4aeb-adef-206ac15312a2\") " pod="openshift-keda/keda-admission-56f9798bb4-2pcbn" Jul 28 16:07:18.941130 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:18.941075 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3fa39b87-81d8-4aeb-adef-206ac15312a2-certificates\") pod \"keda-admission-56f9798bb4-2pcbn\" (UID: \"3fa39b87-81d8-4aeb-adef-206ac15312a2\") " pod="openshift-keda/keda-admission-56f9798bb4-2pcbn" Jul 28 16:07:19.005270 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.005246 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:19.041952 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.041916 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3fa39b87-81d8-4aeb-adef-206ac15312a2-certificates\") pod \"keda-admission-56f9798bb4-2pcbn\" (UID: \"3fa39b87-81d8-4aeb-adef-206ac15312a2\") " pod="openshift-keda/keda-admission-56f9798bb4-2pcbn" Jul 28 16:07:19.042039 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.041976 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfl8d\" (UniqueName: \"kubernetes.io/projected/3fa39b87-81d8-4aeb-adef-206ac15312a2-kube-api-access-kfl8d\") pod \"keda-admission-56f9798bb4-2pcbn\" (UID: \"3fa39b87-81d8-4aeb-adef-206ac15312a2\") " pod="openshift-keda/keda-admission-56f9798bb4-2pcbn" Jul 28 16:07:19.044396 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.044366 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3fa39b87-81d8-4aeb-adef-206ac15312a2-certificates\") pod \"keda-admission-56f9798bb4-2pcbn\" (UID: \"3fa39b87-81d8-4aeb-adef-206ac15312a2\") " pod="openshift-keda/keda-admission-56f9798bb4-2pcbn" Jul 28 16:07:19.054211 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.054188 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfl8d\" (UniqueName: \"kubernetes.io/projected/3fa39b87-81d8-4aeb-adef-206ac15312a2-kube-api-access-kfl8d\") pod \"keda-admission-56f9798bb4-2pcbn\" (UID: \"3fa39b87-81d8-4aeb-adef-206ac15312a2\") " pod="openshift-keda/keda-admission-56f9798bb4-2pcbn" Jul 28 16:07:19.126056 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.126037 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-2pcbn" Jul 28 16:07:19.207435 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.207323 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:19.243090 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.243064 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8494791c-c787-4232-9dad-e6d1afea3823-certificates\") pod \"keda-metrics-apiserver-55557df8c-46x6s\" (UID: \"8494791c-c787-4232-9dad-e6d1afea3823\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" Jul 28 16:07:19.245325 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.245306 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8494791c-c787-4232-9dad-e6d1afea3823-certificates\") pod \"keda-metrics-apiserver-55557df8c-46x6s\" (UID: \"8494791c-c787-4232-9dad-e6d1afea3823\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" Jul 28 16:07:19.254235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.254206 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-2pcbn"] Jul 28 16:07:19.258057 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:07:19.258030 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa39b87_81d8_4aeb_adef_206ac15312a2.slice/crio-cf4886ce60937d001958c369ea79344d87b6fa57ded314d525bb6ef7a8e53088 WatchSource:0}: Error finding container cf4886ce60937d001958c369ea79344d87b6fa57ded314d525bb6ef7a8e53088: Status 404 returned error can't find the container with id cf4886ce60937d001958c369ea79344d87b6fa57ded314d525bb6ef7a8e53088 Jul 28 16:07:19.259485 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.259463 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 16:07:19.306142 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.306118 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:19.405284 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.405254 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" Jul 28 16:07:19.406585 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.406561 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:19.507538 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.507513 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:19.528849 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.528829 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-46x6s"] Jul 28 16:07:19.530890 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:07:19.530865 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8494791c_c787_4232_9dad_e6d1afea3823.slice/crio-4a095957bd0de60831901e3d39dc27eec1999910ac51f093aa0d421f39151923 WatchSource:0}: Error finding container 4a095957bd0de60831901e3d39dc27eec1999910ac51f093aa0d421f39151923: Status 404 returned error can't find the container with id 4a095957bd0de60831901e3d39dc27eec1999910ac51f093aa0d421f39151923 Jul 28 16:07:19.610711 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.610683 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:07:19.680904 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.680873 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" event={"ID":"8494791c-c787-4232-9dad-e6d1afea3823","Type":"ContainerStarted","Data":"4a095957bd0de60831901e3d39dc27eec1999910ac51f093aa0d421f39151923"} Jul 28 16:07:19.681902 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:19.681884 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-2pcbn" event={"ID":"3fa39b87-81d8-4aeb-adef-206ac15312a2","Type":"ContainerStarted","Data":"cf4886ce60937d001958c369ea79344d87b6fa57ded314d525bb6ef7a8e53088"} Jul 28 16:07:22.696364 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:22.696321 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" event={"ID":"8494791c-c787-4232-9dad-e6d1afea3823","Type":"ContainerStarted","Data":"de846061ac31a46897cec1716c5922bd4254b1fd90720533db5f2bc37ca4aca7"} Jul 28 16:07:22.696785 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:22.696388 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" Jul 28 16:07:22.718952 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:22.718892 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" podStartSLOduration=2.496308972 podStartE2EDuration="4.71887568s" totalImagesPullingTime="2.222566708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:07:22.717358752 +0000 UTC m=+310.751526078" watchObservedRunningTime="2026-07-28 16:07:22.71887568 +0000 UTC m=+310.753043007" Jul 28 16:07:24.706529 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:24.706466 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-2pcbn" event={"ID":"3fa39b87-81d8-4aeb-adef-206ac15312a2","Type":"ContainerStarted","Data":"b5bb515315e7d1b9650d391e03960736eda3329845da3db834d7a57b51d445c9"} Jul 28 16:07:24.706941 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:24.706633 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-2pcbn" Jul 28 16:07:24.723705 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:24.723657 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-2pcbn" podStartSLOduration=2.022657236 podStartE2EDuration="6.723640868s" totalImagesPullingTime="4.700983632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:07:24.72223751 +0000 UTC m=+312.756404836" watchObservedRunningTime="2026-07-28 16:07:24.723640868 +0000 UTC m=+312.757808194" Jul 28 16:07:33.705687 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:33.705658 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-46x6s" Jul 28 16:07:45.713047 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:07:45.713007 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-2pcbn" Jul 28 16:08:35.295014 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.294972 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:08:35.381585 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.381560 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-66f44fdc75-s9khk"] Jul 28 16:08:35.384891 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.384871 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" Jul 28 16:08:35.386952 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.386935 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Jul 28 16:08:35.387260 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.387237 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-6rwk9\"" Jul 28 16:08:35.387719 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.387706 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 28 16:08:35.387774 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.387744 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 28 16:08:35.392927 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.392908 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-trdrc"] Jul 28 16:08:35.396206 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.396188 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-66f44fdc75-s9khk"] Jul 28 16:08:35.396295 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.396261 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-654c557856-trdrc" Jul 28 16:08:35.398345 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.398325 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-lpwg6\"" Jul 28 16:08:35.398514 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.398331 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Jul 28 16:08:35.406730 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.406707 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-trdrc"] Jul 28 16:08:35.511219 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.511194 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6af3a443-8a7c-461a-9270-b61909e1ceb6-cert\") pod \"kserve-controller-manager-66f44fdc75-s9khk\" (UID: \"6af3a443-8a7c-461a-9270-b61909e1ceb6\") " pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" Jul 28 16:08:35.511317 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.511225 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt95f\" (UniqueName: \"kubernetes.io/projected/6af3a443-8a7c-461a-9270-b61909e1ceb6-kube-api-access-zt95f\") pod \"kserve-controller-manager-66f44fdc75-s9khk\" (UID: \"6af3a443-8a7c-461a-9270-b61909e1ceb6\") " pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" Jul 28 16:08:35.511317 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.511247 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98a2ff8c-6a69-41da-a8a2-87adc112103f-cert\") pod \"llmisvc-controller-manager-654c557856-trdrc\" (UID: \"98a2ff8c-6a69-41da-a8a2-87adc112103f\") " pod="kserve/llmisvc-controller-manager-654c557856-trdrc" Jul 28 16:08:35.511317 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.511274 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79lbl\" (UniqueName: \"kubernetes.io/projected/98a2ff8c-6a69-41da-a8a2-87adc112103f-kube-api-access-79lbl\") pod \"llmisvc-controller-manager-654c557856-trdrc\" (UID: \"98a2ff8c-6a69-41da-a8a2-87adc112103f\") " pod="kserve/llmisvc-controller-manager-654c557856-trdrc" Jul 28 16:08:35.612111 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.612082 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6af3a443-8a7c-461a-9270-b61909e1ceb6-cert\") pod \"kserve-controller-manager-66f44fdc75-s9khk\" (UID: \"6af3a443-8a7c-461a-9270-b61909e1ceb6\") " pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" Jul 28 16:08:35.612299 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.612122 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt95f\" (UniqueName: \"kubernetes.io/projected/6af3a443-8a7c-461a-9270-b61909e1ceb6-kube-api-access-zt95f\") pod \"kserve-controller-manager-66f44fdc75-s9khk\" (UID: \"6af3a443-8a7c-461a-9270-b61909e1ceb6\") " pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" Jul 28 16:08:35.612299 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.612156 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98a2ff8c-6a69-41da-a8a2-87adc112103f-cert\") pod \"llmisvc-controller-manager-654c557856-trdrc\" (UID: \"98a2ff8c-6a69-41da-a8a2-87adc112103f\") " pod="kserve/llmisvc-controller-manager-654c557856-trdrc" Jul 28 16:08:35.612299 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.612205 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79lbl\" (UniqueName: \"kubernetes.io/projected/98a2ff8c-6a69-41da-a8a2-87adc112103f-kube-api-access-79lbl\") pod \"llmisvc-controller-manager-654c557856-trdrc\" (UID: \"98a2ff8c-6a69-41da-a8a2-87adc112103f\") " pod="kserve/llmisvc-controller-manager-654c557856-trdrc" Jul 28 16:08:35.612299 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:08:35.612213 2550 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 28 16:08:35.612299 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:08:35.612265 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6af3a443-8a7c-461a-9270-b61909e1ceb6-cert podName:6af3a443-8a7c-461a-9270-b61909e1ceb6 nodeName:}" failed. No retries permitted until 2026-07-28 16:08:36.112250749 +0000 UTC m=+384.146418052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6af3a443-8a7c-461a-9270-b61909e1ceb6-cert") pod "kserve-controller-manager-66f44fdc75-s9khk" (UID: "6af3a443-8a7c-461a-9270-b61909e1ceb6") : secret "kserve-webhook-server-cert" not found Jul 28 16:08:35.614693 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.614673 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98a2ff8c-6a69-41da-a8a2-87adc112103f-cert\") pod \"llmisvc-controller-manager-654c557856-trdrc\" (UID: \"98a2ff8c-6a69-41da-a8a2-87adc112103f\") " pod="kserve/llmisvc-controller-manager-654c557856-trdrc" Jul 28 16:08:35.622029 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.622006 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt95f\" (UniqueName: \"kubernetes.io/projected/6af3a443-8a7c-461a-9270-b61909e1ceb6-kube-api-access-zt95f\") pod \"kserve-controller-manager-66f44fdc75-s9khk\" (UID: \"6af3a443-8a7c-461a-9270-b61909e1ceb6\") " pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" Jul 28 16:08:35.622294 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.622272 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79lbl\" (UniqueName: \"kubernetes.io/projected/98a2ff8c-6a69-41da-a8a2-87adc112103f-kube-api-access-79lbl\") pod \"llmisvc-controller-manager-654c557856-trdrc\" (UID: \"98a2ff8c-6a69-41da-a8a2-87adc112103f\") " pod="kserve/llmisvc-controller-manager-654c557856-trdrc" Jul 28 16:08:35.705807 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.705773 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-654c557856-trdrc" Jul 28 16:08:35.835808 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:35.835746 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-trdrc"] Jul 28 16:08:35.838016 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:08:35.837993 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod98a2ff8c_6a69_41da_a8a2_87adc112103f.slice/crio-3d342ce1055a8ba208162d32755e3fa52f16d19b39e6ef2db135cba5581ac1c6 WatchSource:0}: Error finding container 3d342ce1055a8ba208162d32755e3fa52f16d19b39e6ef2db135cba5581ac1c6: Status 404 returned error can't find the container with id 3d342ce1055a8ba208162d32755e3fa52f16d19b39e6ef2db135cba5581ac1c6 Jul 28 16:08:36.015260 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:36.015202 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-trdrc" event={"ID":"98a2ff8c-6a69-41da-a8a2-87adc112103f","Type":"ContainerStarted","Data":"3d342ce1055a8ba208162d32755e3fa52f16d19b39e6ef2db135cba5581ac1c6"} Jul 28 16:08:36.115131 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:36.115108 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6af3a443-8a7c-461a-9270-b61909e1ceb6-cert\") pod \"kserve-controller-manager-66f44fdc75-s9khk\" (UID: \"6af3a443-8a7c-461a-9270-b61909e1ceb6\") " pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" Jul 28 16:08:36.117478 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:36.117452 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6af3a443-8a7c-461a-9270-b61909e1ceb6-cert\") pod \"kserve-controller-manager-66f44fdc75-s9khk\" (UID: \"6af3a443-8a7c-461a-9270-b61909e1ceb6\") " pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" Jul 28 16:08:36.296103 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:36.296035 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" Jul 28 16:08:36.429468 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:36.429436 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-66f44fdc75-s9khk"] Jul 28 16:08:36.431966 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:08:36.431934 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6af3a443_8a7c_461a_9270_b61909e1ceb6.slice/crio-c4e45f10f3ad020a5c681e0d5155122662643969d9c5197bdc057f92baddc56c WatchSource:0}: Error finding container c4e45f10f3ad020a5c681e0d5155122662643969d9c5197bdc057f92baddc56c: Status 404 returned error can't find the container with id c4e45f10f3ad020a5c681e0d5155122662643969d9c5197bdc057f92baddc56c Jul 28 16:08:36.896147 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:36.896117 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:08:37.022808 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:37.022730 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" event={"ID":"6af3a443-8a7c-461a-9270-b61909e1ceb6","Type":"ContainerStarted","Data":"c4e45f10f3ad020a5c681e0d5155122662643969d9c5197bdc057f92baddc56c"} Jul 28 16:08:40.038768 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:40.038721 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-trdrc" event={"ID":"98a2ff8c-6a69-41da-a8a2-87adc112103f","Type":"ContainerStarted","Data":"89f69587dfce2e974b8d4951761c27e108c9d3441fb9f76eab03111c07953ef2"} Jul 28 16:08:40.039231 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:40.038840 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-654c557856-trdrc" Jul 28 16:08:41.044349 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:41.044306 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" event={"ID":"6af3a443-8a7c-461a-9270-b61909e1ceb6","Type":"ContainerStarted","Data":"8654ed6e0435b8f48d200d9bb51888b4c412667348348a4b5c1ad75eaf35a743"} Jul 28 16:08:41.064166 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:41.064121 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" podStartSLOduration=2.23956564 podStartE2EDuration="6.064107424s" totalImagesPullingTime="3.824541784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:08:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:08:41.064067937 +0000 UTC m=+389.098235268" watchObservedRunningTime="2026-07-28 16:08:41.064107424 +0000 UTC m=+389.098274749" Jul 28 16:08:41.065597 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:41.065565 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-654c557856-trdrc" podStartSLOduration=2.8730075189999997 podStartE2EDuration="6.065556455s" totalImagesPullingTime="3.192548936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:08:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:08:40.056804946 +0000 UTC m=+388.090972273" watchObservedRunningTime="2026-07-28 16:08:41.065556455 +0000 UTC m=+389.099723789" Jul 28 16:08:42.048555 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:08:42.048523 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" Jul 28 16:09:09.244397 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:09.244359 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:09:09.315945 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:09.315913 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:09:11.047377 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:11.047340 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-654c557856-trdrc" Jul 28 16:09:13.057677 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:13.057645 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" Jul 28 16:09:18.978807 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:18.978765 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-66f44fdc75-s9khk"] Jul 28 16:09:18.979299 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:18.978973 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" podUID="6af3a443-8a7c-461a-9270-b61909e1ceb6" containerName="manager" containerID="cri-o://8654ed6e0435b8f48d200d9bb51888b4c412667348348a4b5c1ad75eaf35a743" gracePeriod=10 Jul 28 16:09:19.002767 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.002737 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-66f44fdc75-fkdgn"] Jul 28 16:09:19.004944 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.004925 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66f44fdc75-fkdgn" Jul 28 16:09:19.015391 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.015363 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-66f44fdc75-fkdgn"] Jul 28 16:09:19.038904 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.038877 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8kfv\" (UniqueName: \"kubernetes.io/projected/8a5ebedd-f089-4776-b0d2-325e84045576-kube-api-access-r8kfv\") pod \"kserve-controller-manager-66f44fdc75-fkdgn\" (UID: \"8a5ebedd-f089-4776-b0d2-325e84045576\") " pod="kserve/kserve-controller-manager-66f44fdc75-fkdgn" Jul 28 16:09:19.038998 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.038938 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a5ebedd-f089-4776-b0d2-325e84045576-cert\") pod \"kserve-controller-manager-66f44fdc75-fkdgn\" (UID: \"8a5ebedd-f089-4776-b0d2-325e84045576\") " pod="kserve/kserve-controller-manager-66f44fdc75-fkdgn" Jul 28 16:09:19.139527 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.139498 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a5ebedd-f089-4776-b0d2-325e84045576-cert\") pod \"kserve-controller-manager-66f44fdc75-fkdgn\" (UID: \"8a5ebedd-f089-4776-b0d2-325e84045576\") " pod="kserve/kserve-controller-manager-66f44fdc75-fkdgn" Jul 28 16:09:19.139649 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.139559 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8kfv\" (UniqueName: \"kubernetes.io/projected/8a5ebedd-f089-4776-b0d2-325e84045576-kube-api-access-r8kfv\") pod \"kserve-controller-manager-66f44fdc75-fkdgn\" (UID: \"8a5ebedd-f089-4776-b0d2-325e84045576\") " pod="kserve/kserve-controller-manager-66f44fdc75-fkdgn" Jul 28 16:09:19.141640 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.141618 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a5ebedd-f089-4776-b0d2-325e84045576-cert\") pod \"kserve-controller-manager-66f44fdc75-fkdgn\" (UID: \"8a5ebedd-f089-4776-b0d2-325e84045576\") " pod="kserve/kserve-controller-manager-66f44fdc75-fkdgn" Jul 28 16:09:19.150139 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.150113 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8kfv\" (UniqueName: \"kubernetes.io/projected/8a5ebedd-f089-4776-b0d2-325e84045576-kube-api-access-r8kfv\") pod \"kserve-controller-manager-66f44fdc75-fkdgn\" (UID: \"8a5ebedd-f089-4776-b0d2-325e84045576\") " pod="kserve/kserve-controller-manager-66f44fdc75-fkdgn" Jul 28 16:09:19.210224 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.210198 2550 generic.go:356] "Generic (PLEG): container finished" podID="6af3a443-8a7c-461a-9270-b61909e1ceb6" containerID="8654ed6e0435b8f48d200d9bb51888b4c412667348348a4b5c1ad75eaf35a743" exitCode=0 Jul 28 16:09:19.210533 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.210274 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" event={"ID":"6af3a443-8a7c-461a-9270-b61909e1ceb6","Type":"ContainerDied","Data":"8654ed6e0435b8f48d200d9bb51888b4c412667348348a4b5c1ad75eaf35a743"} Jul 28 16:09:19.210533 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.210332 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" event={"ID":"6af3a443-8a7c-461a-9270-b61909e1ceb6","Type":"ContainerDied","Data":"c4e45f10f3ad020a5c681e0d5155122662643969d9c5197bdc057f92baddc56c"} Jul 28 16:09:19.210533 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.210354 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4e45f10f3ad020a5c681e0d5155122662643969d9c5197bdc057f92baddc56c" Jul 28 16:09:19.220230 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.220212 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" Jul 28 16:09:19.342091 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.342067 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zt95f\" (UniqueName: \"kubernetes.io/projected/6af3a443-8a7c-461a-9270-b61909e1ceb6-kube-api-access-zt95f\") pod \"6af3a443-8a7c-461a-9270-b61909e1ceb6\" (UID: \"6af3a443-8a7c-461a-9270-b61909e1ceb6\") " Jul 28 16:09:19.342230 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.342134 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6af3a443-8a7c-461a-9270-b61909e1ceb6-cert\") pod \"6af3a443-8a7c-461a-9270-b61909e1ceb6\" (UID: \"6af3a443-8a7c-461a-9270-b61909e1ceb6\") " Jul 28 16:09:19.344111 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.344085 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6af3a443-8a7c-461a-9270-b61909e1ceb6-kube-api-access-zt95f" (OuterVolumeSpecName: "kube-api-access-zt95f") pod "6af3a443-8a7c-461a-9270-b61909e1ceb6" (UID: "6af3a443-8a7c-461a-9270-b61909e1ceb6"). InnerVolumeSpecName "kube-api-access-zt95f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:09:19.344185 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.344159 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af3a443-8a7c-461a-9270-b61909e1ceb6-cert" (OuterVolumeSpecName: "cert") pod "6af3a443-8a7c-461a-9270-b61909e1ceb6" (UID: "6af3a443-8a7c-461a-9270-b61909e1ceb6"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:09:19.355782 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.355753 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66f44fdc75-fkdgn" Jul 28 16:09:19.442969 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.442938 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt95f\" (UniqueName: \"kubernetes.io/projected/6af3a443-8a7c-461a-9270-b61909e1ceb6-kube-api-access-zt95f\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:09:19.442969 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.442965 2550 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6af3a443-8a7c-461a-9270-b61909e1ceb6-cert\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:09:19.487207 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:19.487102 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-66f44fdc75-fkdgn"] Jul 28 16:09:19.489923 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:09:19.489899 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a5ebedd_f089_4776_b0d2_325e84045576.slice/crio-53f5b3dc34f8b361b4d519ec65863d4e5dd0986d0c8bf6d01e6758a51e41c8ea WatchSource:0}: Error finding container 53f5b3dc34f8b361b4d519ec65863d4e5dd0986d0c8bf6d01e6758a51e41c8ea: Status 404 returned error can't find the container with id 53f5b3dc34f8b361b4d519ec65863d4e5dd0986d0c8bf6d01e6758a51e41c8ea Jul 28 16:09:20.216439 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:20.216385 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66f44fdc75-fkdgn" event={"ID":"8a5ebedd-f089-4776-b0d2-325e84045576","Type":"ContainerStarted","Data":"53f5b3dc34f8b361b4d519ec65863d4e5dd0986d0c8bf6d01e6758a51e41c8ea"} Jul 28 16:09:20.216860 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:20.216475 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66f44fdc75-s9khk" Jul 28 16:09:20.237742 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:20.237718 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-66f44fdc75-s9khk"] Jul 28 16:09:20.241614 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:20.241591 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-66f44fdc75-s9khk"] Jul 28 16:09:20.441125 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:20.441090 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6af3a443-8a7c-461a-9270-b61909e1ceb6" path="/var/lib/kubelet/pods/6af3a443-8a7c-461a-9270-b61909e1ceb6/volumes" Jul 28 16:09:21.222567 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:21.222483 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66f44fdc75-fkdgn" event={"ID":"8a5ebedd-f089-4776-b0d2-325e84045576","Type":"ContainerStarted","Data":"3ee06bb47597d946457b3ea9c6ac031d2e68432e88022c34b5205378484351c7"} Jul 28 16:09:21.222901 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:21.222612 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-66f44fdc75-fkdgn" Jul 28 16:09:21.242282 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:21.242228 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-66f44fdc75-fkdgn" podStartSLOduration=1.8057690819999999 podStartE2EDuration="3.24221371s" totalImagesPullingTime="1.436444628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:09:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:09:21.239998146 +0000 UTC m=+429.274165468" watchObservedRunningTime="2026-07-28 16:09:21.24221371 +0000 UTC m=+429.276381035" Jul 28 16:09:43.364434 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:43.364330 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:09:43.415705 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:43.415673 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:09:43.494133 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:43.494090 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:09:43.550778 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:43.550754 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:09:44.099891 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:44.099863 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:09:44.155794 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:44.155765 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:09:52.232469 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:52.232398 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-66f44fdc75-fkdgn" Jul 28 16:09:53.428955 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:53.428919 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:09:53.530667 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:53.530633 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:09:53.677062 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:53.677023 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:09:53.959276 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:53.959245 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c4f4b4fd9-4zr2p"] Jul 28 16:09:54.025950 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:54.025912 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:09:54.124431 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:54.124386 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:09:54.223745 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:09:54.223685 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:10:12.504035 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:12.503990 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:10:18.985523 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:18.985449 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c4f4b4fd9-4zr2p" podUID="329c6e87-1101-41e3-8be1-e4d8eadd71cf" containerName="console" containerID="cri-o://666d145a1e1ca668d7ce4d7c5f5ef50c76ade296e969a31ed1c3192fee2adece" gracePeriod=15 Jul 28 16:10:19.227530 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.227505 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:10:19.375004 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.374980 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvh77\" (UniqueName: \"kubernetes.io/projected/329c6e87-1101-41e3-8be1-e4d8eadd71cf-kube-api-access-mvh77\") pod \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " Jul 28 16:10:19.375141 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.375019 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-oauth-config\") pod \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " Jul 28 16:10:19.375141 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.375040 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-oauth-serving-cert\") pod \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " Jul 28 16:10:19.375225 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.375149 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-config\") pod \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " Jul 28 16:10:19.375225 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.375218 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-serving-cert\") pod \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " Jul 28 16:10:19.375304 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.375245 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-service-ca\") pod \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " Jul 28 16:10:19.375354 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.375309 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-trusted-ca-bundle\") pod \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\" (UID: \"329c6e87-1101-41e3-8be1-e4d8eadd71cf\") " Jul 28 16:10:19.375438 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.375362 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "329c6e87-1101-41e3-8be1-e4d8eadd71cf" (UID: "329c6e87-1101-41e3-8be1-e4d8eadd71cf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:10:19.375760 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.375593 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-config" (OuterVolumeSpecName: "console-config") pod "329c6e87-1101-41e3-8be1-e4d8eadd71cf" (UID: "329c6e87-1101-41e3-8be1-e4d8eadd71cf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:10:19.375760 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.375621 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-oauth-serving-cert\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:10:19.375760 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.375722 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-service-ca" (OuterVolumeSpecName: "service-ca") pod "329c6e87-1101-41e3-8be1-e4d8eadd71cf" (UID: "329c6e87-1101-41e3-8be1-e4d8eadd71cf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:10:19.375760 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.375754 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "329c6e87-1101-41e3-8be1-e4d8eadd71cf" (UID: "329c6e87-1101-41e3-8be1-e4d8eadd71cf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:10:19.377037 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.377014 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/329c6e87-1101-41e3-8be1-e4d8eadd71cf-kube-api-access-mvh77" (OuterVolumeSpecName: "kube-api-access-mvh77") pod "329c6e87-1101-41e3-8be1-e4d8eadd71cf" (UID: "329c6e87-1101-41e3-8be1-e4d8eadd71cf"). InnerVolumeSpecName "kube-api-access-mvh77". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:10:19.377133 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.377072 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "329c6e87-1101-41e3-8be1-e4d8eadd71cf" (UID: "329c6e87-1101-41e3-8be1-e4d8eadd71cf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:10:19.377177 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.377128 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "329c6e87-1101-41e3-8be1-e4d8eadd71cf" (UID: "329c6e87-1101-41e3-8be1-e4d8eadd71cf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:10:19.476283 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.476257 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-oauth-config\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:10:19.476283 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.476281 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-config\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:10:19.476471 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.476291 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/329c6e87-1101-41e3-8be1-e4d8eadd71cf-console-serving-cert\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:10:19.476471 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.476300 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-service-ca\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:10:19.476471 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.476308 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/329c6e87-1101-41e3-8be1-e4d8eadd71cf-trusted-ca-bundle\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:10:19.476471 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.476316 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvh77\" (UniqueName: \"kubernetes.io/projected/329c6e87-1101-41e3-8be1-e4d8eadd71cf-kube-api-access-mvh77\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:10:19.482701 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.482675 2550 generic.go:356] "Generic (PLEG): container finished" podID="329c6e87-1101-41e3-8be1-e4d8eadd71cf" containerID="666d145a1e1ca668d7ce4d7c5f5ef50c76ade296e969a31ed1c3192fee2adece" exitCode=0 Jul 28 16:10:19.482813 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.482716 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4f4b4fd9-4zr2p" Jul 28 16:10:19.482813 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.482761 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4f4b4fd9-4zr2p" event={"ID":"329c6e87-1101-41e3-8be1-e4d8eadd71cf","Type":"ContainerDied","Data":"666d145a1e1ca668d7ce4d7c5f5ef50c76ade296e969a31ed1c3192fee2adece"} Jul 28 16:10:19.482813 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.482810 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4f4b4fd9-4zr2p" event={"ID":"329c6e87-1101-41e3-8be1-e4d8eadd71cf","Type":"ContainerDied","Data":"d0b7397d5759556aee0441add05f79fb830e14a24673e0ecf2b74281ec3559da"} Jul 28 16:10:19.482933 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.482836 2550 scope.go:117] "RemoveContainer" containerID="666d145a1e1ca668d7ce4d7c5f5ef50c76ade296e969a31ed1c3192fee2adece" Jul 28 16:10:19.491824 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.491805 2550 scope.go:117] "RemoveContainer" containerID="666d145a1e1ca668d7ce4d7c5f5ef50c76ade296e969a31ed1c3192fee2adece" Jul 28 16:10:19.492073 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:10:19.492054 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"666d145a1e1ca668d7ce4d7c5f5ef50c76ade296e969a31ed1c3192fee2adece\": container with ID starting with 666d145a1e1ca668d7ce4d7c5f5ef50c76ade296e969a31ed1c3192fee2adece not found: ID does not exist" containerID="666d145a1e1ca668d7ce4d7c5f5ef50c76ade296e969a31ed1c3192fee2adece" Jul 28 16:10:19.492147 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.492079 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"666d145a1e1ca668d7ce4d7c5f5ef50c76ade296e969a31ed1c3192fee2adece"} err="failed to get container status \"666d145a1e1ca668d7ce4d7c5f5ef50c76ade296e969a31ed1c3192fee2adece\": rpc error: code = NotFound desc = could not find container \"666d145a1e1ca668d7ce4d7c5f5ef50c76ade296e969a31ed1c3192fee2adece\": container with ID starting with 666d145a1e1ca668d7ce4d7c5f5ef50c76ade296e969a31ed1c3192fee2adece not found: ID does not exist" Jul 28 16:10:19.505824 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.505799 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c4f4b4fd9-4zr2p"] Jul 28 16:10:19.509969 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:19.509948 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c4f4b4fd9-4zr2p"] Jul 28 16:10:20.444614 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:10:20.443131 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="329c6e87-1101-41e3-8be1-e4d8eadd71cf" path="/var/lib/kubelet/pods/329c6e87-1101-41e3-8be1-e4d8eadd71cf/volumes" Jul 28 16:11:30.687571 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:11:30.687530 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:11:30.786799 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:11:30.786764 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:11:42.881054 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:11:42.881004 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:11:43.276167 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:11:43.276082 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:12:29.278021 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:12:29.277981 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:12:29.380060 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:12:29.380029 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:13:10.368954 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:13:10.368863 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:13:25.474600 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:13:25.474562 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:13:28.247599 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:13:28.247566 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:13:28.432904 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:13:28.432862 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:13:28.551647 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:13:28.551579 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:13:28.824642 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:13:28.824615 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:13:42.860975 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:13:42.860935 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:13:51.782700 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:13:51.782662 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:14:18.204901 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:14:18.204856 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:14:29.757957 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:14:29.757908 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:15:08.312544 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.312508 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c"] Jul 28 16:15:08.312955 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.312875 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6af3a443-8a7c-461a-9270-b61909e1ceb6" containerName="manager" Jul 28 16:15:08.312955 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.312887 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af3a443-8a7c-461a-9270-b61909e1ceb6" containerName="manager" Jul 28 16:15:08.312955 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.312902 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="329c6e87-1101-41e3-8be1-e4d8eadd71cf" containerName="console" Jul 28 16:15:08.312955 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.312907 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="329c6e87-1101-41e3-8be1-e4d8eadd71cf" containerName="console" Jul 28 16:15:08.313087 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.312958 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="329c6e87-1101-41e3-8be1-e4d8eadd71cf" containerName="console" Jul 28 16:15:08.313087 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.312970 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="6af3a443-8a7c-461a-9270-b61909e1ceb6" containerName="manager" Jul 28 16:15:08.315845 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.315820 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" Jul 28 16:15:08.318036 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.318014 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-8422a-serving-cert\"" Jul 28 16:15:08.318036 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.318023 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 16:15:08.318194 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.318025 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-8422a-kube-rbac-proxy-sar-config\"" Jul 28 16:15:08.318194 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.318090 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-m9t97\"" Jul 28 16:15:08.324513 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.324492 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c"] Jul 28 16:15:08.331559 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.331535 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:15:08.450011 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.449976 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b83a27-d2cf-481c-95d5-b2d139fe559e-openshift-service-ca-bundle\") pod \"model-chainer-raw-8422a-6b8b4b8d84-tk56c\" (UID: \"b8b83a27-d2cf-481c-95d5-b2d139fe559e\") " pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" Jul 28 16:15:08.450011 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.450009 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8b83a27-d2cf-481c-95d5-b2d139fe559e-proxy-tls\") pod \"model-chainer-raw-8422a-6b8b4b8d84-tk56c\" (UID: \"b8b83a27-d2cf-481c-95d5-b2d139fe559e\") " pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" Jul 28 16:15:08.551331 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.551301 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b83a27-d2cf-481c-95d5-b2d139fe559e-openshift-service-ca-bundle\") pod \"model-chainer-raw-8422a-6b8b4b8d84-tk56c\" (UID: \"b8b83a27-d2cf-481c-95d5-b2d139fe559e\") " pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" Jul 28 16:15:08.551331 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.551330 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8b83a27-d2cf-481c-95d5-b2d139fe559e-proxy-tls\") pod \"model-chainer-raw-8422a-6b8b4b8d84-tk56c\" (UID: \"b8b83a27-d2cf-481c-95d5-b2d139fe559e\") " pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" Jul 28 16:15:08.551532 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:15:08.551471 2550 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-raw-8422a-serving-cert: secret "model-chainer-raw-8422a-serving-cert" not found Jul 28 16:15:08.551582 ip-10-0-138-10 kubenswrapper[2550]: E0728 16:15:08.551545 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8b83a27-d2cf-481c-95d5-b2d139fe559e-proxy-tls podName:b8b83a27-d2cf-481c-95d5-b2d139fe559e nodeName:}" failed. No retries permitted until 2026-07-28 16:15:09.051524822 +0000 UTC m=+777.085692139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b8b83a27-d2cf-481c-95d5-b2d139fe559e-proxy-tls") pod "model-chainer-raw-8422a-6b8b4b8d84-tk56c" (UID: "b8b83a27-d2cf-481c-95d5-b2d139fe559e") : secret "model-chainer-raw-8422a-serving-cert" not found Jul 28 16:15:08.551943 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.551925 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b83a27-d2cf-481c-95d5-b2d139fe559e-openshift-service-ca-bundle\") pod \"model-chainer-raw-8422a-6b8b4b8d84-tk56c\" (UID: \"b8b83a27-d2cf-481c-95d5-b2d139fe559e\") " pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" Jul 28 16:15:08.848071 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:08.848040 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:15:09.054913 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:09.054881 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8b83a27-d2cf-481c-95d5-b2d139fe559e-proxy-tls\") pod \"model-chainer-raw-8422a-6b8b4b8d84-tk56c\" (UID: \"b8b83a27-d2cf-481c-95d5-b2d139fe559e\") " pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" Jul 28 16:15:09.057237 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:09.057207 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8b83a27-d2cf-481c-95d5-b2d139fe559e-proxy-tls\") pod \"model-chainer-raw-8422a-6b8b4b8d84-tk56c\" (UID: \"b8b83a27-d2cf-481c-95d5-b2d139fe559e\") " pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" Jul 28 16:15:09.227145 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:09.227073 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" Jul 28 16:15:09.361281 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:09.361246 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c"] Jul 28 16:15:09.364731 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:15:09.364702 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8b83a27_d2cf_481c_95d5_b2d139fe559e.slice/crio-67a022b567646f4827afa317261393af1372e5874bb37fb8f35d684856f39d58 WatchSource:0}: Error finding container 67a022b567646f4827afa317261393af1372e5874bb37fb8f35d684856f39d58: Status 404 returned error can't find the container with id 67a022b567646f4827afa317261393af1372e5874bb37fb8f35d684856f39d58 Jul 28 16:15:09.366601 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:09.366581 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 16:15:09.717730 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:09.717694 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" event={"ID":"b8b83a27-d2cf-481c-95d5-b2d139fe559e","Type":"ContainerStarted","Data":"67a022b567646f4827afa317261393af1372e5874bb37fb8f35d684856f39d58"} Jul 28 16:15:12.392540 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:12.392510 2550 scope.go:117] "RemoveContainer" containerID="8654ed6e0435b8f48d200d9bb51888b4c412667348348a4b5c1ad75eaf35a743" Jul 28 16:15:12.735158 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:12.735088 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" event={"ID":"b8b83a27-d2cf-481c-95d5-b2d139fe559e","Type":"ContainerStarted","Data":"50d799fd57edd2b2c5b1b4705626cae2261577e4060901bcfea70784f454594f"} Jul 28 16:15:12.735274 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:12.735162 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" Jul 28 16:15:12.752884 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:12.752838 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" podStartSLOduration=2.251747369 podStartE2EDuration="4.752824247s" totalImagesPullingTime="2.501076878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:15:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:15:12.751373945 +0000 UTC m=+780.785541272" watchObservedRunningTime="2026-07-28 16:15:12.752824247 +0000 UTC m=+780.786991573" Jul 28 16:15:18.362948 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:18.362915 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c"] Jul 28 16:15:18.363460 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:18.363098 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" podUID="b8b83a27-d2cf-481c-95d5-b2d139fe559e" containerName="model-chainer-raw-8422a" containerID="cri-o://50d799fd57edd2b2c5b1b4705626cae2261577e4060901bcfea70784f454594f" gracePeriod=30 Jul 28 16:15:18.368762 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:18.368728 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" podUID="b8b83a27-d2cf-481c-95d5-b2d139fe559e" containerName="model-chainer-raw-8422a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:15:18.396900 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:18.396871 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:15:18.686574 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:18.686492 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:15:18.849518 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:18.849474 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:15:18.930941 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:18.930912 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:15:18.998437 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:18.998348 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:15:19.140964 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:19.140933 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:15:23.367238 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:23.367186 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" podUID="b8b83a27-d2cf-481c-95d5-b2d139fe559e" containerName="model-chainer-raw-8422a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:15:28.367018 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:28.366974 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" podUID="b8b83a27-d2cf-481c-95d5-b2d139fe559e" containerName="model-chainer-raw-8422a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:15:33.367541 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:33.367494 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" podUID="b8b83a27-d2cf-481c-95d5-b2d139fe559e" containerName="model-chainer-raw-8422a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:15:38.367475 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:38.367360 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" podUID="b8b83a27-d2cf-481c-95d5-b2d139fe559e" containerName="model-chainer-raw-8422a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:15:43.367789 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:43.367736 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" podUID="b8b83a27-d2cf-481c-95d5-b2d139fe559e" containerName="model-chainer-raw-8422a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:15:48.367516 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:48.367464 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" podUID="b8b83a27-d2cf-481c-95d5-b2d139fe559e" containerName="model-chainer-raw-8422a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:15:48.889714 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:48.889661 2550 generic.go:356] "Generic (PLEG): container finished" podID="b8b83a27-d2cf-481c-95d5-b2d139fe559e" containerID="50d799fd57edd2b2c5b1b4705626cae2261577e4060901bcfea70784f454594f" exitCode=0 Jul 28 16:15:48.889847 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:48.889741 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" event={"ID":"b8b83a27-d2cf-481c-95d5-b2d139fe559e","Type":"ContainerDied","Data":"50d799fd57edd2b2c5b1b4705626cae2261577e4060901bcfea70784f454594f"} Jul 28 16:15:49.020957 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:49.020934 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" Jul 28 16:15:49.031101 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:49.031079 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8b83a27-d2cf-481c-95d5-b2d139fe559e-proxy-tls\") pod \"b8b83a27-d2cf-481c-95d5-b2d139fe559e\" (UID: \"b8b83a27-d2cf-481c-95d5-b2d139fe559e\") " Jul 28 16:15:49.031220 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:49.031164 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b83a27-d2cf-481c-95d5-b2d139fe559e-openshift-service-ca-bundle\") pod \"b8b83a27-d2cf-481c-95d5-b2d139fe559e\" (UID: \"b8b83a27-d2cf-481c-95d5-b2d139fe559e\") " Jul 28 16:15:49.031509 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:49.031485 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8b83a27-d2cf-481c-95d5-b2d139fe559e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b8b83a27-d2cf-481c-95d5-b2d139fe559e" (UID: "b8b83a27-d2cf-481c-95d5-b2d139fe559e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:15:49.033097 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:49.033077 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b83a27-d2cf-481c-95d5-b2d139fe559e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b8b83a27-d2cf-481c-95d5-b2d139fe559e" (UID: "b8b83a27-d2cf-481c-95d5-b2d139fe559e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:15:49.131777 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:49.131725 2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8b83a27-d2cf-481c-95d5-b2d139fe559e-openshift-service-ca-bundle\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:15:49.131777 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:49.131745 2550 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8b83a27-d2cf-481c-95d5-b2d139fe559e-proxy-tls\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:15:49.895735 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:49.895705 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" Jul 28 16:15:49.896159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:49.895751 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c" event={"ID":"b8b83a27-d2cf-481c-95d5-b2d139fe559e","Type":"ContainerDied","Data":"67a022b567646f4827afa317261393af1372e5874bb37fb8f35d684856f39d58"} Jul 28 16:15:49.896159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:49.895810 2550 scope.go:117] "RemoveContainer" containerID="50d799fd57edd2b2c5b1b4705626cae2261577e4060901bcfea70784f454594f" Jul 28 16:15:49.916285 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:49.916261 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c"] Jul 28 16:15:49.918326 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:49.918306 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-8422a-6b8b4b8d84-tk56c"] Jul 28 16:15:50.442981 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:15:50.442948 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8b83a27-d2cf-481c-95d5-b2d139fe559e" path="/var/lib/kubelet/pods/b8b83a27-d2cf-481c-95d5-b2d139fe559e/volumes" Jul 28 16:16:58.710228 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.710193 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7"] Jul 28 16:16:58.710720 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.710580 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b83a27-d2cf-481c-95d5-b2d139fe559e" containerName="model-chainer-raw-8422a" Jul 28 16:16:58.710720 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.710595 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b83a27-d2cf-481c-95d5-b2d139fe559e" containerName="model-chainer-raw-8422a" Jul 28 16:16:58.710720 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.710651 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b83a27-d2cf-481c-95d5-b2d139fe559e" containerName="model-chainer-raw-8422a" Jul 28 16:16:58.713639 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.713619 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" Jul 28 16:16:58.716464 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.716442 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-69837-serving-cert\"" Jul 28 16:16:58.716686 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.716665 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-69837-kube-rbac-proxy-sar-config\"" Jul 28 16:16:58.716760 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.716694 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-m9t97\"" Jul 28 16:16:58.717037 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.717024 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 16:16:58.724462 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.724390 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7"] Jul 28 16:16:58.738916 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.738882 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:16:58.865294 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.865257 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:16:58.872389 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.872363 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04d5f33b-db61-4962-b711-5ff2ff545212-proxy-tls\") pod \"model-chainer-raw-hpa-69837-7887cbffbd-r62r7\" (UID: \"04d5f33b-db61-4962-b711-5ff2ff545212\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" Jul 28 16:16:58.872514 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.872493 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04d5f33b-db61-4962-b711-5ff2ff545212-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-69837-7887cbffbd-r62r7\" (UID: \"04d5f33b-db61-4962-b711-5ff2ff545212\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" Jul 28 16:16:58.973867 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.973780 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04d5f33b-db61-4962-b711-5ff2ff545212-proxy-tls\") pod \"model-chainer-raw-hpa-69837-7887cbffbd-r62r7\" (UID: \"04d5f33b-db61-4962-b711-5ff2ff545212\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" Jul 28 16:16:58.973987 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.973886 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04d5f33b-db61-4962-b711-5ff2ff545212-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-69837-7887cbffbd-r62r7\" (UID: \"04d5f33b-db61-4962-b711-5ff2ff545212\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" Jul 28 16:16:58.974625 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.974607 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04d5f33b-db61-4962-b711-5ff2ff545212-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-69837-7887cbffbd-r62r7\" (UID: \"04d5f33b-db61-4962-b711-5ff2ff545212\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" Jul 28 16:16:58.976320 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:58.976299 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04d5f33b-db61-4962-b711-5ff2ff545212-proxy-tls\") pod \"model-chainer-raw-hpa-69837-7887cbffbd-r62r7\" (UID: \"04d5f33b-db61-4962-b711-5ff2ff545212\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" Jul 28 16:16:59.046470 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:59.046430 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" Jul 28 16:16:59.172347 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:59.172291 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7"] Jul 28 16:16:59.175000 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:16:59.174972 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d5f33b_db61_4962_b711_5ff2ff545212.slice/crio-b0c513e15061092de7ddde17694284d6b9df5ee114a67790ae5b4b7e3e33efd4 WatchSource:0}: Error finding container b0c513e15061092de7ddde17694284d6b9df5ee114a67790ae5b4b7e3e33efd4: Status 404 returned error can't find the container with id b0c513e15061092de7ddde17694284d6b9df5ee114a67790ae5b4b7e3e33efd4 Jul 28 16:16:59.181736 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:16:59.181712 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" event={"ID":"04d5f33b-db61-4962-b711-5ff2ff545212","Type":"ContainerStarted","Data":"b0c513e15061092de7ddde17694284d6b9df5ee114a67790ae5b4b7e3e33efd4"} Jul 28 16:17:00.187547 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:00.187507 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" event={"ID":"04d5f33b-db61-4962-b711-5ff2ff545212","Type":"ContainerStarted","Data":"a03b25e49bc87bcb02c47c2e56e1fbb7808cd44733878b7173920c66d0d3ff4f"} Jul 28 16:17:00.187985 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:00.187578 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" Jul 28 16:17:00.204968 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:00.204925 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" podStartSLOduration=2.204909826 podStartE2EDuration="2.204909826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:17:00.203617125 +0000 UTC m=+888.237784456" watchObservedRunningTime="2026-07-28 16:17:00.204909826 +0000 UTC m=+888.239077152" Jul 28 16:17:06.198741 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:06.198653 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" Jul 28 16:17:08.769324 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:08.769291 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7"] Jul 28 16:17:08.769728 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:08.769479 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" podUID="04d5f33b-db61-4962-b711-5ff2ff545212" containerName="model-chainer-raw-hpa-69837" containerID="cri-o://a03b25e49bc87bcb02c47c2e56e1fbb7808cd44733878b7173920c66d0d3ff4f" gracePeriod=30 Jul 28 16:17:08.791738 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:08.791708 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:17:08.980758 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:08.980728 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:17:09.067553 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:09.067519 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:17:09.144741 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:09.144707 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:17:09.312988 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:09.312958 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:17:11.196911 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:11.196868 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" podUID="04d5f33b-db61-4962-b711-5ff2ff545212" containerName="model-chainer-raw-hpa-69837" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:17:16.196584 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:16.196526 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" podUID="04d5f33b-db61-4962-b711-5ff2ff545212" containerName="model-chainer-raw-hpa-69837" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:17:21.196537 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:21.196493 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" podUID="04d5f33b-db61-4962-b711-5ff2ff545212" containerName="model-chainer-raw-hpa-69837" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:17:21.196924 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:21.196591 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" Jul 28 16:17:26.196060 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:26.196011 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" podUID="04d5f33b-db61-4962-b711-5ff2ff545212" containerName="model-chainer-raw-hpa-69837" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:17:28.999446 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:28.999381 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:17:29.397474 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:29.397439 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:17:31.196296 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:31.196258 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" podUID="04d5f33b-db61-4962-b711-5ff2ff545212" containerName="model-chainer-raw-hpa-69837" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:17:36.196092 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:36.196052 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" podUID="04d5f33b-db61-4962-b711-5ff2ff545212" containerName="model-chainer-raw-hpa-69837" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:17:39.354545 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:39.354494 2550 generic.go:356] "Generic (PLEG): container finished" podID="04d5f33b-db61-4962-b711-5ff2ff545212" containerID="a03b25e49bc87bcb02c47c2e56e1fbb7808cd44733878b7173920c66d0d3ff4f" exitCode=0 Jul 28 16:17:39.354984 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:39.354574 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" event={"ID":"04d5f33b-db61-4962-b711-5ff2ff545212","Type":"ContainerDied","Data":"a03b25e49bc87bcb02c47c2e56e1fbb7808cd44733878b7173920c66d0d3ff4f"} Jul 28 16:17:39.417904 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:39.417874 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" Jul 28 16:17:39.469532 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:39.469509 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04d5f33b-db61-4962-b711-5ff2ff545212-openshift-service-ca-bundle\") pod \"04d5f33b-db61-4962-b711-5ff2ff545212\" (UID: \"04d5f33b-db61-4962-b711-5ff2ff545212\") " Jul 28 16:17:39.469658 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:39.469547 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04d5f33b-db61-4962-b711-5ff2ff545212-proxy-tls\") pod \"04d5f33b-db61-4962-b711-5ff2ff545212\" (UID: \"04d5f33b-db61-4962-b711-5ff2ff545212\") " Jul 28 16:17:39.469826 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:39.469802 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04d5f33b-db61-4962-b711-5ff2ff545212-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "04d5f33b-db61-4962-b711-5ff2ff545212" (UID: "04d5f33b-db61-4962-b711-5ff2ff545212"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:17:39.471567 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:39.471542 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04d5f33b-db61-4962-b711-5ff2ff545212-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "04d5f33b-db61-4962-b711-5ff2ff545212" (UID: "04d5f33b-db61-4962-b711-5ff2ff545212"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:17:39.570439 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:39.570401 2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04d5f33b-db61-4962-b711-5ff2ff545212-openshift-service-ca-bundle\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:17:39.570538 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:39.570442 2550 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/04d5f33b-db61-4962-b711-5ff2ff545212-proxy-tls\") on node \"ip-10-0-138-10.ec2.internal\" DevicePath \"\"" Jul 28 16:17:40.359435 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:40.359394 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" event={"ID":"04d5f33b-db61-4962-b711-5ff2ff545212","Type":"ContainerDied","Data":"b0c513e15061092de7ddde17694284d6b9df5ee114a67790ae5b4b7e3e33efd4"} Jul 28 16:17:40.359801 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:40.359436 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7" Jul 28 16:17:40.359801 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:40.359465 2550 scope.go:117] "RemoveContainer" containerID="a03b25e49bc87bcb02c47c2e56e1fbb7808cd44733878b7173920c66d0d3ff4f" Jul 28 16:17:40.381430 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:40.381392 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7"] Jul 28 16:17:40.385305 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:40.385284 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-69837-7887cbffbd-r62r7"] Jul 28 16:17:40.441352 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:17:40.441328 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04d5f33b-db61-4962-b711-5ff2ff545212" path="/var/lib/kubelet/pods/04d5f33b-db61-4962-b711-5ff2ff545212/volumes" Jul 28 16:19:04.343326 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:19:04.343289 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:19:06.487865 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:19:06.487819 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:54.437464 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:20:54.437426 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:54.635642 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:21:54.635562 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:59.721131 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:21:59.721065 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:59.806343 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:21:59.806309 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:59.878238 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:21:59.878204 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:00.387001 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:22:00.386960 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:11.961626 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:22:11.961581 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:23:29.993610 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:23:29.993555 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:23:30.067422 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:23:30.067381 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:23:35.048164 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:23:35.048124 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:23:35.128273 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:23:35.128238 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:23:35.245799 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:23:35.245768 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:23:35.340033 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:23:35.340003 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:25:05.256725 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:25:05.256649 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:26:55.779923 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:26:55.779882 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:27:00.664230 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:27:00.664197 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:27:00.776382 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:27:00.776319 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:27:00.881544 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:27:00.881514 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:27:00.945544 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:27:00.945483 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:27:14.689686 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:27:14.689640 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:28:08.577636 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:28:08.577590 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:28:15.034801 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:28:15.034756 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:28:30.799650 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:28:30.799585 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:30:01.072240 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:30:01.072185 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:30:06.188485 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:30:06.188445 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:30:06.279953 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:30:06.279917 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:30:06.353904 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:30:06.353873 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:30:06.460952 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:30:06.460862 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:31:36.347229 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:31:36.347191 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:31:38.592302 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:31:38.592271 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:31:46.462184 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:31:46.462134 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:31:46.603106 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:31:46.603074 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:31.755976 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:33:31.755939 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:31.835845 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:33:31.835795 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:49.876434 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:33:49.876331 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:49.960678 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:33:49.960633 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:55.041479 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:33:55.041444 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:55.389931 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:33:55.389897 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:34:00.138635 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:34:00.138575 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:34:05.304202 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:34:05.304145 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:34:05.432583 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:34:05.432550 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:35:35.499050 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:35:35.499014 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:35:40.589343 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:35:40.589313 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:35:40.671507 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:35:40.671470 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:37:10.923845 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:10.923751 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:37:16.864806 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:16.864771 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:37:16.950546 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:16.950508 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:37:34.456840 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:34.456805 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:37:46.979712 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:46.979669 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:37:47.209965 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:47.209924 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:37:47.321235 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:47.321203 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:37:50.201783 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:50.201741 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nww4s_676a8d84-5dd4-4207-91e6-a8256cdfa19e/global-pull-secret-syncer/0.log" Jul 28 16:37:50.324633 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:50.324607 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" Jul 28 16:37:50.470454 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:50.470373 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-10.ec2.internal_ce1d00f21bc867d7fa218ae3a0c694e1/haproxy/0.log" Jul 28 16:37:53.284642 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:53.284602 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:37:53.347184 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:53.347153 2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9nkv2_50a3000c-4028-4746-81fa-99633b072bb5/konnectivity-agent/0.log" limit=1024 Jul 28 16:37:54.908288 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:54.908251 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38a6f237-40ab-4a17-98b5-b7d66c8e81dd/alertmanager/0.log" Jul 28 16:37:54.995370 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:54.995339 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38a6f237-40ab-4a17-98b5-b7d66c8e81dd/config-reloader/0.log" Jul 28 16:37:55.057047 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:55.057012 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38a6f237-40ab-4a17-98b5-b7d66c8e81dd/kube-rbac-proxy-web/0.log" Jul 28 16:37:55.112910 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:55.112888 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38a6f237-40ab-4a17-98b5-b7d66c8e81dd/kube-rbac-proxy/0.log" Jul 28 16:37:55.154020 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:55.153993 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38a6f237-40ab-4a17-98b5-b7d66c8e81dd/kube-rbac-proxy-metric/0.log" Jul 28 16:37:55.190387 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:55.190328 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38a6f237-40ab-4a17-98b5-b7d66c8e81dd/prom-label-proxy/0.log" Jul 28 16:37:55.227256 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:55.227234 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38a6f237-40ab-4a17-98b5-b7d66c8e81dd/init-config-reloader/0.log" Jul 28 16:37:55.314188 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:55.314170 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-lx4v7_2f7c7d70-020c-493d-987d-ec6e3291368d/kube-state-metrics/0.log" Jul 28 16:37:55.361425 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:55.361383 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-lx4v7_2f7c7d70-020c-493d-987d-ec6e3291368d/kube-rbac-proxy-main/0.log" Jul 28 16:37:55.404822 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:55.404796 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-lx4v7_2f7c7d70-020c-493d-987d-ec6e3291368d/kube-rbac-proxy-self/0.log" Jul 28 16:37:55.492881 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:55.492812 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-xn5jk_f26a3188-b900-4f3a-88d3-2f9a856e48b7/monitoring-plugin/0.log" Jul 28 16:37:55.655197 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:55.655134 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m8bzl_8b930b8e-8a7a-4045-8bc9-2e45b0b3c073/node-exporter/0.log" Jul 28 16:37:55.684241 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:55.684218 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m8bzl_8b930b8e-8a7a-4045-8bc9-2e45b0b3c073/kube-rbac-proxy/0.log" Jul 28 16:37:55.718526 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:55.718497 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m8bzl_8b930b8e-8a7a-4045-8bc9-2e45b0b3c073/init-textfile/0.log" Jul 28 16:37:56.479064 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:56.478987 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-95fz5_f377f0bf-3bdd-41f5-bedc-b5259079ff6f/prometheus-operator-admission-webhook/0.log" Jul 28 16:37:56.668112 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:56.668086 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c878fcd8d-tp2ct_3616399a-0634-4f02-90ef-58950b10598b/thanos-query/0.log" Jul 28 16:37:56.706812 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:56.706794 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c878fcd8d-tp2ct_3616399a-0634-4f02-90ef-58950b10598b/kube-rbac-proxy-web/0.log" Jul 28 16:37:56.753728 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:56.753677 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c878fcd8d-tp2ct_3616399a-0634-4f02-90ef-58950b10598b/kube-rbac-proxy/0.log" Jul 28 16:37:56.788189 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:56.788172 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c878fcd8d-tp2ct_3616399a-0634-4f02-90ef-58950b10598b/prom-label-proxy/0.log" Jul 28 16:37:56.832597 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:56.832576 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c878fcd8d-tp2ct_3616399a-0634-4f02-90ef-58950b10598b/kube-rbac-proxy-rules/0.log" Jul 28 16:37:56.873738 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:56.873719 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c878fcd8d-tp2ct_3616399a-0634-4f02-90ef-58950b10598b/kube-rbac-proxy-metrics/0.log" Jul 28 16:37:57.865762 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:57.865718 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz"] Jul 28 16:37:57.866173 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:57.866120 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04d5f33b-db61-4962-b711-5ff2ff545212" containerName="model-chainer-raw-hpa-69837" Jul 28 16:37:57.866173 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:57.866140 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="04d5f33b-db61-4962-b711-5ff2ff545212" containerName="model-chainer-raw-hpa-69837" Jul 28 16:37:57.866271 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:57.866255 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="04d5f33b-db61-4962-b711-5ff2ff545212" containerName="model-chainer-raw-hpa-69837" Jul 28 16:37:57.869488 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:57.869467 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:57.871860 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:57.871834 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-j7d85\"/\"default-dockercfg-zs968\"" Jul 28 16:37:57.872549 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:57.872529 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j7d85\"/\"kube-root-ca.crt\"" Jul 28 16:37:57.872645 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:57.872560 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j7d85\"/\"openshift-service-ca.crt\"" Jul 28 16:37:57.881320 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:57.881295 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz"] Jul 28 16:37:58.004459 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.004436 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba13b441-6bc0-4d4b-b901-579f3af028b7-proc\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.004573 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.004491 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sxtj\" (UniqueName: \"kubernetes.io/projected/ba13b441-6bc0-4d4b-b901-579f3af028b7-kube-api-access-6sxtj\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.004573 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.004534 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba13b441-6bc0-4d4b-b901-579f3af028b7-sys\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.004662 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.004577 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba13b441-6bc0-4d4b-b901-579f3af028b7-podres\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.004662 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.004610 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba13b441-6bc0-4d4b-b901-579f3af028b7-lib-modules\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.105692 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.105662 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba13b441-6bc0-4d4b-b901-579f3af028b7-podres\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.105807 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.105697 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba13b441-6bc0-4d4b-b901-579f3af028b7-lib-modules\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.105807 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.105762 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba13b441-6bc0-4d4b-b901-579f3af028b7-proc\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.105918 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.105813 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sxtj\" (UniqueName: \"kubernetes.io/projected/ba13b441-6bc0-4d4b-b901-579f3af028b7-kube-api-access-6sxtj\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.105918 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.105830 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ba13b441-6bc0-4d4b-b901-579f3af028b7-podres\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.105918 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.105842 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba13b441-6bc0-4d4b-b901-579f3af028b7-sys\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.105918 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.105868 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ba13b441-6bc0-4d4b-b901-579f3af028b7-proc\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.105918 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.105912 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba13b441-6bc0-4d4b-b901-579f3af028b7-lib-modules\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.106155 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.105924 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba13b441-6bc0-4d4b-b901-579f3af028b7-sys\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.117100 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.117047 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sxtj\" (UniqueName: \"kubernetes.io/projected/ba13b441-6bc0-4d4b-b901-579f3af028b7-kube-api-access-6sxtj\") pod \"perf-node-gather-daemonset-8rbnz\" (UID: \"ba13b441-6bc0-4d4b-b901-579f3af028b7\") " pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.179587 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.179568 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:58.320174 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:58.320151 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz"] Jul 28 16:37:58.322234 ip-10-0-138-10 kubenswrapper[2550]: W0728 16:37:58.322199 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podba13b441_6bc0_4d4b_b901_579f3af028b7.slice/crio-8e1c23e244796cab40ad0ec91fc4c28d1c8bea7fa4560670ecdffce5e937f491 WatchSource:0}: Error finding container 8e1c23e244796cab40ad0ec91fc4c28d1c8bea7fa4560670ecdffce5e937f491: Status 404 returned error can't find the container with id 8e1c23e244796cab40ad0ec91fc4c28d1c8bea7fa4560670ecdffce5e937f491 Jul 28 16:37:59.303646 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:59.303613 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" event={"ID":"ba13b441-6bc0-4d4b-b901-579f3af028b7","Type":"ContainerStarted","Data":"a29767f73158c454feb9303e7c8440c945d51b92b2994a61c573cf566bc5c715"} Jul 28 16:37:59.304007 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:59.303666 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" event={"ID":"ba13b441-6bc0-4d4b-b901-579f3af028b7","Type":"ContainerStarted","Data":"8e1c23e244796cab40ad0ec91fc4c28d1c8bea7fa4560670ecdffce5e937f491"} Jul 28 16:37:59.304007 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:59.303699 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:37:59.332915 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:37:59.329943 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" podStartSLOduration=2.329922481 podStartE2EDuration="2.329922481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:37:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:37:59.32976558 +0000 UTC m=+2147.363932908" watchObservedRunningTime="2026-07-28 16:37:59.329922481 +0000 UTC m=+2147.364089821" Jul 28 16:38:00.806269 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:00.806236 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-28lcc_d4ecf78f-a024-4986-888d-67b82853a16b/dns/0.log" Jul 28 16:38:00.834363 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:00.834336 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-28lcc_d4ecf78f-a024-4986-888d-67b82853a16b/kube-rbac-proxy/0.log" Jul 28 16:38:01.092710 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:01.092680 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jn6q8_afa6eae0-4d40-4f40-9358-69feb4831e42/dns-node-resolver/0.log" Jul 28 16:38:01.638350 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:01.638319 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-757d4dcc58-mqzbm_eca560de-d0ef-4627-81db-12d0636ced3f/registry/0.log" Jul 28 16:38:01.693624 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:01.693601 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pdxnl_088caeed-2739-4bd0-836b-3a7b0e456194/node-ca/0.log" Jul 28 16:38:02.608159 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:02.608129 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7f859bc589-qkbls_6c1c9a74-abad-430a-ad3f-fe0d70833ca6/router/0.log" Jul 28 16:38:02.995674 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:02.995584 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b5jtv_dd3769ac-7034-4a90-9100-03479a58e78a/serve-healthcheck-canary/0.log" Jul 28 16:38:03.464796 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:03.464766 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-lxlwg_8f454402-1f88-4231-a0f5-5bc2f58d0a01/insights-operator/0.log" Jul 28 16:38:03.465282 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:03.465261 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-lxlwg_8f454402-1f88-4231-a0f5-5bc2f58d0a01/insights-operator/1.log" Jul 28 16:38:03.490939 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:03.490916 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8r9q6_6084bd94-c631-4f0f-a5a5-869566f1b430/kube-rbac-proxy/0.log" Jul 28 16:38:03.517792 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:03.517772 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8r9q6_6084bd94-c631-4f0f-a5a5-869566f1b430/exporter/0.log" Jul 28 16:38:03.547388 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:03.547368 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8r9q6_6084bd94-c631-4f0f-a5a5-869566f1b430/extractor/0.log" Jul 28 16:38:05.318245 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:05.318168 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-j7d85/perf-node-gather-daemonset-8rbnz" Jul 28 16:38:06.163042 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:06.163010 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-66f44fdc75-fkdgn_8a5ebedd-f089-4776-b0d2-325e84045576/manager/0.log" Jul 28 16:38:06.198030 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:06.198000 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-654c557856-trdrc_98a2ff8c-6a69-41da-a8a2-87adc112103f/manager/0.log" Jul 28 16:38:11.282617 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:11.282587 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-hwml6_eae2599b-c9ae-4d7e-b5c6-7db936009ab8/migrator/0.log" Jul 28 16:38:11.317624 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:11.317603 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-hwml6_eae2599b-c9ae-4d7e-b5c6-7db936009ab8/graceful-termination/0.log" Jul 28 16:38:12.810031 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:12.809995 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-88478_b46c2d3e-bf32-4f6a-ab84-6e84d1a6abbb/kube-multus/0.log" Jul 28 16:38:12.844305 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:12.844275 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2ldwp_05a91de5-b17a-40da-96f0-1a0a25ca6c93/kube-multus-additional-cni-plugins/0.log" Jul 28 16:38:12.873786 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:12.873760 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2ldwp_05a91de5-b17a-40da-96f0-1a0a25ca6c93/egress-router-binary-copy/0.log" Jul 28 16:38:12.908674 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:12.908646 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2ldwp_05a91de5-b17a-40da-96f0-1a0a25ca6c93/cni-plugins/0.log" Jul 28 16:38:12.941769 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:12.941747 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2ldwp_05a91de5-b17a-40da-96f0-1a0a25ca6c93/bond-cni-plugin/0.log" Jul 28 16:38:12.971648 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:12.971628 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2ldwp_05a91de5-b17a-40da-96f0-1a0a25ca6c93/routeoverride-cni/0.log" Jul 28 16:38:13.004903 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:13.004880 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2ldwp_05a91de5-b17a-40da-96f0-1a0a25ca6c93/whereabouts-cni-bincopy/0.log" Jul 28 16:38:13.035013 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:13.034992 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2ldwp_05a91de5-b17a-40da-96f0-1a0a25ca6c93/whereabouts-cni/0.log" Jul 28 16:38:13.581367 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:13.581337 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ngzg7_ddb31891-1896-4c34-97c8-6e45e42d349c/network-metrics-daemon/0.log" Jul 28 16:38:13.611573 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:13.611543 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ngzg7_ddb31891-1896-4c34-97c8-6e45e42d349c/kube-rbac-proxy/0.log" Jul 28 16:38:14.588231 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:14.588198 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xnwq_5d397052-5e4d-4ba0-b66a-d05699dfdebf/ovn-controller/0.log" Jul 28 16:38:14.615978 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:14.615948 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xnwq_5d397052-5e4d-4ba0-b66a-d05699dfdebf/ovn-acl-logging/0.log" Jul 28 16:38:14.649855 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:14.649824 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xnwq_5d397052-5e4d-4ba0-b66a-d05699dfdebf/kube-rbac-proxy-node/0.log" Jul 28 16:38:14.686166 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:14.686128 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xnwq_5d397052-5e4d-4ba0-b66a-d05699dfdebf/kube-rbac-proxy-ovn-metrics/0.log" Jul 28 16:38:14.716924 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:14.716892 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xnwq_5d397052-5e4d-4ba0-b66a-d05699dfdebf/northd/0.log" Jul 28 16:38:14.748038 ip-10-0-138-10 kubenswrapper[2550]: I0728 16:38:14.748011 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4xnwq_5d397052-5e4d-4ba0-b66a-d05699dfdebf/nbdb/0.log"