Jul 09 18:01:47.309213 ip-10-0-137-168 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 18:01:47.309225 ip-10-0-137-168 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 18:01:47.309232 ip-10-0-137-168 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 18:01:47.309474 ip-10-0-137-168 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 18:01:57.427741 ip-10-0-137-168 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 09 18:01:57.427757 ip-10-0-137-168 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 46fe5e63404241fabbab0b4388b63a90 -- Jul 09 18:04:03.396966 ip-10-0-137-168 systemd[1]: Starting Kubernetes Kubelet... Jul 09 18:04:03.874918 ip-10-0-137-168 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 18:04:03.874918 ip-10-0-137-168 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 18:04:03.874918 ip-10-0-137-168 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 18:04:03.874918 ip-10-0-137-168 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 18:04:03.874918 ip-10-0-137-168 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 18:04:03.876035 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.874981 2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 18:04:03.878629 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878612 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:04:03.878629 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878628 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878632 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878636 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878639 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878642 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878645 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878650 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878654 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878657 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878660 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878663 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878675 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878678 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878680 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878696 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878700 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878703 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878705 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878708 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:04:03.878710 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878718 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878721 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878724 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878727 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878729 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878732 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878734 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878736 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878739 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878741 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878744 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878746 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878749 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878751 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878753 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878756 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878758 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878761 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878763 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878765 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:04:03.879152 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878767 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878770 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878773 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878775 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878778 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878780 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878782 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878785 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878787 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878790 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878793 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878795 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878800 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878804 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878807 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878811 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878814 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878816 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878819 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878822 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:04:03.879652 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878825 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878830 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878833 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878836 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878838 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878842 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878845 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878847 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878850 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878852 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878855 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878858 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878861 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878864 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878866 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878869 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878871 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878873 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878876 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:04:03.880166 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878878 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878881 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878883 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878885 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878888 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878891 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878893 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878897 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878899 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878902 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878904 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878907 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878910 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878912 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878914 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878917 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878919 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878922 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878924 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.878926 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:04:03.880624 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879601 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879608 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879611 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879613 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879616 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879619 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879622 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879624 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879627 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879629 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879632 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879634 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879637 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879639 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879642 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879644 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879647 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879649 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879651 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879655 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:04:03.881122 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879658 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879661 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879664 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879666 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879669 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879671 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879674 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879676 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879680 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879699 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879702 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879705 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879708 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879710 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879713 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879715 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879718 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879721 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879724 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:04:03.881646 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879727 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879729 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879732 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879735 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879737 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879739 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879742 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879744 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879747 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879750 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879752 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879755 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879758 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879761 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879763 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879784 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879788 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879791 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879793 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879796 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:04:03.882143 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879800 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879804 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879807 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879810 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879812 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879815 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879818 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879820 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879822 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879825 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879827 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879830 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879833 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879837 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879839 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879842 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879844 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879847 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879849 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879852 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:04:03.882630 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879854 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879857 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879859 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879862 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879866 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879870 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879872 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879876 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879879 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879881 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879884 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879887 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879890 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879892 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879895 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879897 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879900 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879902 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879905 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:04:03.883134 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.879907 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880273 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880277 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880280 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880283 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880286 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880288 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880291 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880294 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880296 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880299 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880301 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880304 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880306 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880311 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880314 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880318 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880321 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880324 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:04:03.883594 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880326 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880330 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880333 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880336 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880339 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880342 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880344 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880347 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880350 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880352 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880355 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880358 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880361 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880364 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880367 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880370 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880372 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880375 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880377 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:04:03.884076 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880380 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880382 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880385 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880387 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880391 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880393 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880395 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880398 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880400 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880402 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880405 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880411 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880414 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880416 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880418 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880421 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880423 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880426 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880428 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880431 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:04:03.884532 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880434 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880438 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880440 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880443 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880446 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880448 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880451 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880453 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880456 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880458 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880461 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880463 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880466 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880468 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880471 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880474 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880477 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880479 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880482 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880484 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:04:03.885040 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880487 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880489 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880492 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880496 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880498 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880501 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880503 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880506 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880508 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880511 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880513 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880516 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880518 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880521 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880523 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880525 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880527 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880530 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880532 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880535 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:04:03.885525 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880537 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.880540 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880614 2569 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880621 2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880627 2569 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880631 2569 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880636 2569 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880639 2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880643 2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880647 2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880650 2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880653 2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880656 2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880659 2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880662 2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880667 2569 flags.go:64] FLAG: --cgroup-root="" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880669 2569 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880672 2569 flags.go:64] FLAG: --client-ca-file="" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880675 2569 flags.go:64] FLAG: --cloud-provider="external" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880678 2569 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880699 2569 flags.go:64] FLAG: --cluster-domain="" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880703 2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880706 2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880709 2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 18:04:03.886074 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880712 2569 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880716 2569 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880719 2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880722 2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880725 2569 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880728 2569 flags.go:64] FLAG: --contention-profiling="false" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880731 2569 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880734 2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880737 2569 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880740 2569 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880744 2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880747 2569 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880749 2569 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880752 2569 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880755 2569 flags.go:64] FLAG: --enable-server="true" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880758 2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880762 2569 flags.go:64] FLAG: --event-burst="100" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880765 2569 flags.go:64] FLAG: --event-qps="50" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880768 2569 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880771 2569 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880773 2569 flags.go:64] FLAG: --eviction-hard="" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880777 2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880781 2569 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880786 2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880788 2569 flags.go:64] FLAG: --eviction-soft="" Jul 09 18:04:03.886641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880791 2569 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880794 2569 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880797 2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880799 2569 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880802 2569 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880805 2569 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880807 2569 flags.go:64] FLAG: --feature-gates="" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880811 2569 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880814 2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880817 2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880820 2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880823 2569 flags.go:64] FLAG: --healthz-port="10248" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880825 2569 flags.go:64] FLAG: --help="false" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880828 2569 flags.go:64] FLAG: --hostname-override="ip-10-0-137-168.ec2.internal" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880831 2569 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880834 2569 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880837 2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880840 2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880843 2569 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880846 2569 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880849 2569 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880851 2569 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880854 2569 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 18:04:03.887252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880857 2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880860 2569 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880863 2569 flags.go:64] FLAG: --kube-reserved="" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880865 2569 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880868 2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880871 2569 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880875 2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880879 2569 flags.go:64] FLAG: --lock-file="" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880882 2569 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880885 2569 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880888 2569 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880892 2569 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880895 2569 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880898 2569 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880900 2569 flags.go:64] FLAG: --logging-format="text" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880903 2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880906 2569 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880909 2569 flags.go:64] FLAG: --manifest-url="" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880911 2569 flags.go:64] FLAG: --manifest-url-header="" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880915 2569 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880918 2569 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880922 2569 flags.go:64] FLAG: --max-pods="110" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880924 2569 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880927 2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880930 2569 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 18:04:03.887895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880932 2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880935 2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880938 2569 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880941 2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880948 2569 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880951 2569 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880954 2569 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880956 2569 flags.go:64] FLAG: --pod-cidr="" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880960 2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880965 2569 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880968 2569 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880971 2569 flags.go:64] FLAG: --pods-per-core="0" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880974 2569 flags.go:64] FLAG: --port="10250" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880978 2569 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880980 2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07566796aa3695544" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880985 2569 flags.go:64] FLAG: --qos-reserved="" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880988 2569 flags.go:64] FLAG: --read-only-port="10255" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880991 2569 flags.go:64] FLAG: --register-node="true" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880993 2569 flags.go:64] FLAG: --register-with-taints="" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.880997 2569 flags.go:64] FLAG: --registry-burst="10" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881000 2569 flags.go:64] FLAG: --registry-qps="5" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881002 2569 flags.go:64] FLAG: --reserved-cpus="" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881005 2569 flags.go:64] FLAG: --reserved-memory="" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881009 2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881011 2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 18:04:03.888494 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881015 2569 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881017 2569 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881020 2569 flags.go:64] FLAG: --runonce="false" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881023 2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881026 2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881029 2569 flags.go:64] FLAG: --seccomp-default="false" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881032 2569 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881034 2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881037 2569 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881040 2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881043 2569 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881046 2569 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881049 2569 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881051 2569 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881054 2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881057 2569 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881060 2569 flags.go:64] FLAG: --system-cgroups="" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881062 2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881067 2569 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881070 2569 flags.go:64] FLAG: --tls-cert-file="" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881074 2569 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881077 2569 flags.go:64] FLAG: --tls-min-version="" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881081 2569 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881084 2569 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881087 2569 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 18:04:03.889112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881089 2569 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881092 2569 flags.go:64] FLAG: --v="2" Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881096 2569 flags.go:64] FLAG: --version="false" Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881099 2569 flags.go:64] FLAG: --vmodule="" Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881103 2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.881106 2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881218 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881221 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881224 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881227 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881230 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881232 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881235 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881237 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881240 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881242 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881245 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881247 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881249 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881252 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881254 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881257 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:04:03.889721 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881260 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881262 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881265 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881267 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881270 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881274 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881276 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881281 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881283 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881285 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881288 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881291 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881293 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881295 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881298 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881300 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881303 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881305 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881307 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881310 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:04:03.890253 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881312 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881314 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881317 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881319 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881322 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881324 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881326 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881329 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881332 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881334 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881336 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881339 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881341 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881343 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881346 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881348 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881351 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881355 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881358 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881363 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:04:03.890753 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881367 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881369 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881372 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881375 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881377 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881379 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881382 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881384 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881387 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881389 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881392 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881394 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881396 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881399 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881401 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881403 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881406 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881408 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881411 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881413 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:04:03.891264 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881415 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881418 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881420 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881422 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881425 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881427 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881429 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881433 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881437 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881441 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881444 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881448 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881451 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881454 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881456 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881459 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881461 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881464 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881466 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:04:03.891752 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881469 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:04:03.892213 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881471 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:04:03.892213 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881474 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:04:03.892213 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.881478 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:04:03.892213 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.882139 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 18:04:03.892213 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.889141 2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 09 18:04:03.892213 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.889161 2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 18:04:03.892213 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.890084 2569 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 18:04:03.892213 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.890092 2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 18:04:03.892213 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890157 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:04:03.892213 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890163 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:04:03.892213 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890166 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:04:03.892213 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890169 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:04:03.892213 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890172 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:04:03.892213 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890175 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890178 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890181 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890184 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890187 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890190 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890193 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890203 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890206 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890209 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890211 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890214 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890216 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890219 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890221 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890224 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890226 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890230 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890234 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890237 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:04:03.892560 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890240 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890243 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890246 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890251 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890254 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890258 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890261 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890263 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890266 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890269 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890272 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890274 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890276 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890279 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890282 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890284 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890287 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890289 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890292 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:04:03.893070 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890294 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890302 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890305 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890307 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890310 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890313 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890315 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890318 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890320 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890322 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890325 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890327 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890329 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890332 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890334 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890337 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890340 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890342 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890345 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890347 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:04:03.893582 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890349 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890352 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890354 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890356 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890359 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890361 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890365 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890368 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890371 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890374 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890376 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890379 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890382 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890384 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890392 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890395 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890397 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890400 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890402 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:04:03.894087 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890405 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890408 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890410 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890413 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890415 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890418 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890420 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890422 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890425 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890427 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890430 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890432 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890435 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890437 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890440 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:04:03.894554 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890442 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.890447 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890609 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890614 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890617 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890620 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890623 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890626 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890628 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890631 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890633 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890636 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890638 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890646 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890649 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 18:04:03.895021 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890651 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890654 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890656 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890659 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890662 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890674 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890679 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890682 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890702 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890705 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890707 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890710 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890713 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890715 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890718 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890720 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890724 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890729 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890732 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890734 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 18:04:03.895392 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890737 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890739 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890742 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890744 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890746 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890749 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890751 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890754 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890756 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890759 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890761 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890770 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890773 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890776 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890778 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890781 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890790 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890792 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890795 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890797 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 18:04:03.895881 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890800 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890802 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890805 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890808 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890811 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890813 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890816 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890818 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890820 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890823 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890825 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890827 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890830 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890832 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890835 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890837 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890839 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890842 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890845 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 18:04:03.896365 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890848 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890851 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890854 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890857 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890859 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890870 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890874 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890876 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890879 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890882 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890885 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890888 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890890 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890893 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890896 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890898 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890901 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890904 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890906 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890909 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 18:04:03.896823 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890911 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 18:04:03.897588 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890914 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 18:04:03.897588 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890916 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 18:04:03.897588 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890919 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 18:04:03.897588 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890921 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 18:04:03.897588 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890923 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 09 18:04:03.897588 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:03.890926 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 18:04:03.897588 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.890931 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 18:04:03.897588 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.891048 2569 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 18:04:03.897588 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.893266 2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 18:04:03.897588 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.894832 2569 server.go:1019] "Starting client certificate rotation" Jul 09 18:04:03.897588 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.894943 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 18:04:03.897588 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.894979 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 18:04:03.906987 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.906963 2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 18:04:03.909277 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.909255 2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 18:04:03.921442 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.921425 2569 log.go:25] "Validated CRI v1 runtime API" Jul 09 18:04:03.925285 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.925266 2569 log.go:25] "Validated CRI v1 image API" Jul 09 18:04:03.926119 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.926104 2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 18:04:03.929216 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.929190 2569 fs.go:135] Filesystem UUIDs: map[65512779-1c2b-4fa0-a2b7-af72c9fe3371:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c75eba2c-07ba-4c25-b0f7-13569a9a3b3d:/dev/nvme0n1p4] Jul 09 18:04:03.929273 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.929216 2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 18:04:03.931399 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.931383 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 18:04:03.935866 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.935751 2569 manager.go:216] Machine: {Timestamp:2026-07-09 18:04:03.933343906 +0000 UTC m=+0.398866309 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100071 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29525ee87edec7e45776aca86354dd SystemUUID:ec29525e-e87e-dec7-e457-76aca86354dd BootID:46fe5e63-4042-41fa-bbab-0b4388b63a90 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dc:f4:13:25:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dc:f4:13:25:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:95:64:ec:f7:3c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 18:04:03.935866 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.935860 2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 18:04:03.935988 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.935973 2569 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 18:04:03.939483 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.939457 2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 18:04:03.939668 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.939485 2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-168.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 18:04:03.940932 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.940919 2569 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 18:04:03.940997 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.940938 2569 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 18:04:03.940997 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.940948 2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 18:04:03.940997 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.940977 2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 18:04:03.942422 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.942410 2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 18:04:03.945380 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.945366 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 09 18:04:03.945806 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.945667 2569 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 18:04:03.945871 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.945833 2569 kubelet.go:486] "Attempting to sync node with API server" Jul 09 18:04:03.945871 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.945859 2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 18:04:03.945978 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.945877 2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 18:04:03.945978 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.945891 2569 kubelet.go:398] "Adding apiserver pod source" Jul 09 18:04:03.945978 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.945902 2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 18:04:03.946409 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.946393 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 18:04:03.946487 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.946426 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 18:04:03.949680 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.949658 2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 09 18:04:03.951616 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.951602 2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 18:04:03.951672 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.951632 2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 18:04:03.951824 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.951814 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 18:04:03.951859 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.951829 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 18:04:03.951859 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.951834 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 18:04:03.951859 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.951839 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 18:04:03.951859 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.951847 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 18:04:03.951859 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.951852 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 18:04:03.952437 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.952429 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 18:04:03.952468 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.952438 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 18:04:03.952468 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.952444 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 18:04:03.952468 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.952449 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 18:04:03.952468 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.952457 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 18:04:03.952468 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.952465 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 18:04:03.953435 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.953425 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 18:04:03.953435 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.953435 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 18:04:03.956976 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.956963 2569 server.go:1268] "Started kubelet" Jul 09 18:04:03.957167 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.957092 2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 18:04:03.957218 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.957161 2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-168.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 18:04:03.957218 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:03.957158 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 18:04:03.957314 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:03.957262 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 18:04:03.957436 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.957382 2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 18:04:03.957551 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.957460 2569 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 18:04:03.958447 ip-10-0-137-168 systemd[1]: Started Kubernetes Kubelet. Jul 09 18:04:03.958578 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.958536 2569 server.go:310] "Adding debug handlers to kubelet server" Jul 09 18:04:03.958644 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.958598 2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 18:04:03.962330 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:03.962306 2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 18:04:03.964777 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.964760 2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 18:04:03.964777 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.964768 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 18:04:03.964953 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:03.964840 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:03.965425 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.965411 2569 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 18:04:03.965425 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.965414 2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 18:04:03.965425 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.965427 2569 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 18:04:03.965670 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.965654 2569 reconstruct.go:97] "Volume reconstruction finished" Jul 09 18:04:03.965670 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.965670 2569 reconciler.go:29] "Reconciler: start to sync state" Jul 09 18:04:03.966076 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.966042 2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 18:04:03.966076 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.966070 2569 factory.go:55] Registering systemd factory Jul 09 18:04:03.966076 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.966080 2569 factory.go:223] Registration of the systemd container factory successfully Jul 09 18:04:03.966319 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.966301 2569 factory.go:153] Registering CRI-O factory Jul 09 18:04:03.966372 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.966325 2569 factory.go:223] Registration of the crio container factory successfully Jul 09 18:04:03.966372 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.966369 2569 factory.go:103] Registering Raw factory Jul 09 18:04:03.966462 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.966385 2569 manager.go:1196] Started watching for new ooms in manager Jul 09 18:04:03.966887 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:03.965578 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-168.ec2.internal.18c0b1296f3fcbba default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-168.ec2.internal,UID:ip-10-0-137-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-168.ec2.internal,},FirstTimestamp:2026-07-09 18:04:03.956943802 +0000 UTC m=+0.422466203,LastTimestamp:2026-07-09 18:04:03.956943802 +0000 UTC m=+0.422466203,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-168.ec2.internal,}" Jul 09 18:04:03.967094 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.967067 2569 manager.go:349] Starting recovery of all containers Jul 09 18:04:03.976769 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:03.976446 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 18:04:03.976769 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:03.976611 2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 18:04:03.978261 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.978238 2569 manager.go:354] Recovery completed Jul 09 18:04:03.979308 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.979282 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9nkpl" Jul 09 18:04:03.980238 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.980116 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 18:04:03.983945 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.983772 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:04:03.985717 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.985680 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:04:03.985780 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.985736 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:04:03.985780 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.985752 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:04:03.986262 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.986247 2569 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 18:04:03.986262 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.986260 2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 18:04:03.986380 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.986276 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 09 18:04:03.986380 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.986302 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:04:03.986997 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.986983 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:04:03.987060 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.987008 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:04:03.987060 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.987023 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:04:03.987639 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.987628 2569 policy_none.go:49] "None policy: Start" Jul 09 18:04:03.987676 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.987646 2569 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 18:04:03.987676 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.987657 2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 18:04:03.988598 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.988588 2569 policy_none.go:47] "Start" Jul 09 18:04:03.994935 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:03.994920 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9nkpl" Jul 09 18:04:03.997483 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:03.997394 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-168.ec2.internal.18c0b12970f6de2d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-168.ec2.internal,UID:ip-10-0-137-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-168.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-168.ec2.internal,},FirstTimestamp:2026-07-09 18:04:03.985718829 +0000 UTC m=+0.451241239,LastTimestamp:2026-07-09 18:04:03.985718829 +0000 UTC m=+0.451241239,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-168.ec2.internal,}" Jul 09 18:04:04.026225 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.026211 2569 manager.go:337] "Starting Device Plugin manager" Jul 09 18:04:04.026325 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.026243 2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 18:04:04.026325 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.026257 2569 server.go:87] "Starting device plugin registration server" Jul 09 18:04:04.026487 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.026474 2569 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 18:04:04.026543 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.026488 2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 18:04:04.026619 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.026572 2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 18:04:04.026677 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.026648 2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 18:04:04.026677 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.026658 2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 18:04:04.027248 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.027228 2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 18:04:04.027317 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.027300 2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:04.126970 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.126899 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:04:04.128098 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.128068 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:04:04.128204 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.128117 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:04:04.128204 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.128133 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:04:04.128204 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.128171 2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.135140 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.135119 2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.135225 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.135148 2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-168.ec2.internal\": node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:04.160997 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.160972 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:04.190648 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.190613 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 18:04:04.190648 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.190657 2569 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 18:04:04.190846 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.190700 2569 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 18:04:04.190846 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.190741 2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 18:04:04.192953 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.192929 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:04:04.261979 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.261945 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:04.291681 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.291652 2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal"] Jul 09 18:04:04.291781 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.291761 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:04:04.292837 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.292815 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:04:04.292964 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.292852 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:04:04.292964 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.292867 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:04:04.294254 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.294239 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:04:04.294409 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.294387 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.294467 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.294437 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:04:04.295024 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.295001 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:04:04.295106 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.295002 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:04:04.295106 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.295064 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:04:04.295106 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.295076 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:04:04.295106 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.295038 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:04:04.295319 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.295118 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:04:04.296240 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.296227 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.296359 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.296270 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:04:04.297011 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.296988 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:04:04.297097 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.297019 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:04:04.297097 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.297035 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:04:04.325693 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.325674 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-168.ec2.internal\" not found" node="ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.330598 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.330583 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-168.ec2.internal\" not found" node="ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.362030 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.362002 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:04.367795 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.367770 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/631442f90a9aec34b0d279ea0f5f9c73-config\") pod \"kube-apiserver-proxy-ip-10-0-137-168.ec2.internal\" (UID: \"631442f90a9aec34b0d279ea0f5f9c73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.367916 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.367810 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6fca482dcad147ae6c62a690d3cba064-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal\" (UID: \"6fca482dcad147ae6c62a690d3cba064\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.367916 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.367830 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fca482dcad147ae6c62a690d3cba064-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal\" (UID: \"6fca482dcad147ae6c62a690d3cba064\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.462387 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.462307 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:04.467928 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.467905 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fca482dcad147ae6c62a690d3cba064-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal\" (UID: \"6fca482dcad147ae6c62a690d3cba064\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.467988 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.467937 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/631442f90a9aec34b0d279ea0f5f9c73-config\") pod \"kube-apiserver-proxy-ip-10-0-137-168.ec2.internal\" (UID: \"631442f90a9aec34b0d279ea0f5f9c73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.467988 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.467960 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6fca482dcad147ae6c62a690d3cba064-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal\" (UID: \"6fca482dcad147ae6c62a690d3cba064\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.468058 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.468013 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/631442f90a9aec34b0d279ea0f5f9c73-config\") pod \"kube-apiserver-proxy-ip-10-0-137-168.ec2.internal\" (UID: \"631442f90a9aec34b0d279ea0f5f9c73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.468058 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.468014 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fca482dcad147ae6c62a690d3cba064-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal\" (UID: \"6fca482dcad147ae6c62a690d3cba064\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.468121 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.468054 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6fca482dcad147ae6c62a690d3cba064-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal\" (UID: \"6fca482dcad147ae6c62a690d3cba064\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.569384 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.569356 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:04.628840 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.628800 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.633895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.633875 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" Jul 09 18:04:04.670574 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.670540 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:04.771883 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.771839 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:04.815416 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.815383 2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:04:04.872117 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.872088 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:04.894594 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.894568 2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 18:04:04.895160 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.894749 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 18:04:04.895160 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.894748 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 18:04:04.972432 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:04.972402 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:04.998396 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.998360 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 17:59:04 +0000 UTC" deadline="2028-04-09 02:09:29.366734414 +0000 UTC" Jul 09 18:04:04.998396 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:04.998394 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15344h5m24.368344413s" Jul 09 18:04:05.073195 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:05.073129 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:05.123755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.123728 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 18:04:05.126950 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.126927 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientMemory" Jul 09 18:04:05.127143 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.126971 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 18:04:05.127143 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.126991 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeHasSufficientPID" Jul 09 18:04:05.173525 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:05.173496 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:05.231229 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:05.231194 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod631442f90a9aec34b0d279ea0f5f9c73.slice/crio-89555452aed2a17ed2074c814dbaabc039c27bb7c9284a413f7fd6e780b523b4 WatchSource:0}: Error finding container 89555452aed2a17ed2074c814dbaabc039c27bb7c9284a413f7fd6e780b523b4: Status 404 returned error can't find the container with id 89555452aed2a17ed2074c814dbaabc039c27bb7c9284a413f7fd6e780b523b4 Jul 09 18:04:05.231953 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:05.231930 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fca482dcad147ae6c62a690d3cba064.slice/crio-a956048cb7c2789aba0b4fabcebd98412c6f4073274fa3c4f2479228a556f694 WatchSource:0}: Error finding container a956048cb7c2789aba0b4fabcebd98412c6f4073274fa3c4f2479228a556f694: Status 404 returned error can't find the container with id a956048cb7c2789aba0b4fabcebd98412c6f4073274fa3c4f2479228a556f694 Jul 09 18:04:05.238792 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.238775 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 18:04:05.273838 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:05.273816 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:05.311211 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.311192 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:04:05.374457 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:05.374404 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:05.475174 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:05.475146 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-168.ec2.internal\" not found" Jul 09 18:04:05.550467 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.550438 2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:04:05.565261 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.565237 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" Jul 09 18:04:05.581299 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.581276 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 18:04:05.582663 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.582638 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" Jul 09 18:04:05.595398 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.595380 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 18:04:05.948454 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.948423 2569 apiserver.go:52] "Watching apiserver" Jul 09 18:04:05.959625 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.959598 2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 18:04:05.960100 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.960076 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-cr5qg","openshift-network-operator/iptables-alerter-hspnc","openshift-ovn-kubernetes/ovnkube-node-sjxbv","kube-system/konnectivity-agent-2qp88","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal","openshift-multus/multus-additional-cni-plugins-xvhp7","openshift-multus/multus-tkxzm","openshift-network-diagnostics/network-check-target-lntgg","kube-system/global-pull-secret-syncer-sq92v","kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal","openshift-cluster-node-tuning-operator/tuned-kwgh9","openshift-dns/node-resolver-klp4q","openshift-image-registry/node-ca-zs6xt"] Jul 09 18:04:05.962606 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.962586 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:05.964852 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.964826 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-klp4q" Jul 09 18:04:05.965741 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.965718 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 18:04:05.965854 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.965828 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 18:04:05.965941 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.965922 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 18:04:05.966004 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.965964 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gqfpp\"" Jul 09 18:04:05.966004 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.965924 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 18:04:05.966107 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.965993 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 18:04:05.966289 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.966269 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 18:04:05.967923 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.967900 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.970116 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.970095 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2qp88" Jul 09 18:04:05.972249 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.971934 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 18:04:05.972998 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.972976 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 18:04:05.973090 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.973041 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xptgc\"" Jul 09 18:04:05.973151 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.973137 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 18:04:05.973276 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.973250 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 18:04:05.973347 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.973276 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 18:04:05.973401 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.973367 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 18:04:05.973401 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.973379 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 18:04:05.973498 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.973281 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 18:04:05.973619 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.973600 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 18:04:05.973677 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.973602 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-w7ns2\"" Jul 09 18:04:05.973733 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.973722 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-l545p\"" Jul 09 18:04:05.973802 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.973790 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 18:04:05.975719 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.975681 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:05.976786 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.976753 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-run-systemd\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.976867 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.976795 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-var-lib-openvswitch\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.976867 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.976822 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-log-socket\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.976867 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.976844 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02d39fbc-ea0c-43d7-a221-f825998aa6d8-ovnkube-script-lib\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.977031 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.976868 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d084e88e-cb6f-4344-9d93-fee760d1d9c3-system-cni-dir\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:05.977031 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.976907 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d084e88e-cb6f-4344-9d93-fee760d1d9c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:05.977031 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.976955 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d084e88e-cb6f-4344-9d93-fee760d1d9c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:05.977031 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977012 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02d39fbc-ea0c-43d7-a221-f825998aa6d8-env-overrides\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.977242 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977048 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d084e88e-cb6f-4344-9d93-fee760d1d9c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:05.977242 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977089 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-etc-openvswitch\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.977242 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977115 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-cni-netd\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.977242 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977141 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8s49\" (UniqueName: \"kubernetes.io/projected/0a8abea8-0e83-46f9-9fc1-f22428d89987-kube-api-access-f8s49\") pod \"node-resolver-klp4q\" (UID: \"0a8abea8-0e83-46f9-9fc1-f22428d89987\") " pod="openshift-dns/node-resolver-klp4q" Jul 09 18:04:05.977242 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977166 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-run-openvswitch\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.977242 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977191 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-node-log\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.977242 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977217 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bw6c\" (UniqueName: \"kubernetes.io/projected/d084e88e-cb6f-4344-9d93-fee760d1d9c3-kube-api-access-9bw6c\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:05.977590 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977244 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a8abea8-0e83-46f9-9fc1-f22428d89987-tmp-dir\") pod \"node-resolver-klp4q\" (UID: \"0a8abea8-0e83-46f9-9fc1-f22428d89987\") " pod="openshift-dns/node-resolver-klp4q" Jul 09 18:04:05.977590 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977311 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-kubelet\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.977590 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977355 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-run-ovn-kubernetes\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.977590 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977382 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.977590 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977419 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02d39fbc-ea0c-43d7-a221-f825998aa6d8-ovn-node-metrics-cert\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.977590 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977450 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0a8abea8-0e83-46f9-9fc1-f22428d89987-hosts-file\") pod \"node-resolver-klp4q\" (UID: \"0a8abea8-0e83-46f9-9fc1-f22428d89987\") " pod="openshift-dns/node-resolver-klp4q" Jul 09 18:04:05.977590 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977477 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-slash\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.977590 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977515 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-run-netns\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.977590 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977556 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02d39fbc-ea0c-43d7-a221-f825998aa6d8-ovnkube-config\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.978057 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977611 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-run-ovn\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.978057 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977645 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-cni-bin\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.978057 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977677 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d084e88e-cb6f-4344-9d93-fee760d1d9c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:05.978057 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977813 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-systemd-units\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.978057 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977854 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnfls\" (UniqueName: \"kubernetes.io/projected/02d39fbc-ea0c-43d7-a221-f825998aa6d8-kube-api-access-nnfls\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:05.978057 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977886 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d084e88e-cb6f-4344-9d93-fee760d1d9c3-cnibin\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:05.978057 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.977911 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d084e88e-cb6f-4344-9d93-fee760d1d9c3-os-release\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:05.978320 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.978128 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zs6xt" Jul 09 18:04:05.981037 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.981014 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:05.981116 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:05.981089 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:05.981423 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.981400 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 18:04:05.981544 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.981518 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7cn7m\"" Jul 09 18:04:05.981618 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.981545 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 18:04:05.981751 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.981716 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 18:04:05.982001 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.981988 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 18:04:05.982263 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.982242 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 18:04:05.982454 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.982436 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 18:04:05.982616 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.982597 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tsfs8\"" Jul 09 18:04:05.991816 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.990956 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:05.991816 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:05.991059 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:05.991816 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.991588 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 18:04:05.994178 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.994150 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hspnc" Jul 09 18:04:05.996990 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.996970 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:05.997133 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:05.997109 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:05.998234 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.998165 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rxm4h\"" Jul 09 18:04:05.998356 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.998243 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:04:05.998356 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.998315 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 18:04:05.999115 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.999097 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 18:04:05.999304 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:05.999286 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.001889 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.001870 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.002282 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.002255 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gshv2\"" Jul 09 18:04:06.002370 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.002357 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 18:04:06.006872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.006662 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 18:04:06.007047 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.007031 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xtbcc\"" Jul 09 18:04:06.007972 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.007906 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 18:04:06.052175 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.052148 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9d7wr" Jul 09 18:04:06.064381 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.064358 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9d7wr" Jul 09 18:04:06.066965 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.066934 2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 18:04:06.079231 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079208 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0a8abea8-0e83-46f9-9fc1-f22428d89987-hosts-file\") pod \"node-resolver-klp4q\" (UID: \"0a8abea8-0e83-46f9-9fc1-f22428d89987\") " pod="openshift-dns/node-resolver-klp4q" Jul 09 18:04:06.079352 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079236 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-slash\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.079352 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079255 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-run-netns\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.079352 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079282 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq4d7\" (UniqueName: \"kubernetes.io/projected/3112e5e6-80ab-4355-b4cc-669776e66483-kube-api-access-rq4d7\") pod \"iptables-alerter-hspnc\" (UID: \"3112e5e6-80ab-4355-b4cc-669776e66483\") " pod="openshift-network-operator/iptables-alerter-hspnc" Jul 09 18:04:06.079352 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079340 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0a8abea8-0e83-46f9-9fc1-f22428d89987-hosts-file\") pod \"node-resolver-klp4q\" (UID: \"0a8abea8-0e83-46f9-9fc1-f22428d89987\") " pod="openshift-dns/node-resolver-klp4q" Jul 09 18:04:06.079577 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079365 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjffd\" (UniqueName: \"kubernetes.io/projected/c4cada06-9af5-4483-a7a4-eca67319ff23-kube-api-access-pjffd\") pod \"node-ca-zs6xt\" (UID: \"c4cada06-9af5-4483-a7a4-eca67319ff23\") " pod="openshift-image-registry/node-ca-zs6xt" Jul 09 18:04:06.079577 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079341 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-run-netns\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.079577 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079403 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qqrb\" (UniqueName: \"kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb\") pod \"network-check-target-lntgg\" (UID: \"aa5fad78-82ce-49bb-a50e-2b178586bdaf\") " pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:06.079577 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079341 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-slash\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.079577 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079425 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3fa7022d-512b-4625-95bb-1342a17d197f-agent-certs\") pod \"konnectivity-agent-2qp88\" (UID: \"3fa7022d-512b-4625-95bb-1342a17d197f\") " pod="kube-system/konnectivity-agent-2qp88" Jul 09 18:04:06.079577 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079442 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-var-lib-cni-bin\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.079577 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079462 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-hostroot\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.079577 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079515 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3fa7022d-512b-4625-95bb-1342a17d197f-konnectivity-ca\") pod \"konnectivity-agent-2qp88\" (UID: \"3fa7022d-512b-4625-95bb-1342a17d197f\") " pod="kube-system/konnectivity-agent-2qp88" Jul 09 18:04:06.079577 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079533 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-run-k8s-cni-cncf-io\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.079577 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079557 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-var-lib-cni-multus\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079585 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d084e88e-cb6f-4344-9d93-fee760d1d9c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079602 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3112e5e6-80ab-4355-b4cc-669776e66483-host-slash\") pod \"iptables-alerter-hspnc\" (UID: \"3112e5e6-80ab-4355-b4cc-669776e66483\") " pod="openshift-network-operator/iptables-alerter-hspnc" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079621 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-modprobe-d\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079644 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-sysctl-d\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079707 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-log-socket\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079744 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9c90246-157e-41aa-9498-a676922f8f0f-cni-binary-copy\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079772 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9c90246-157e-41aa-9498-a676922f8f0f-multus-daemon-config\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079789 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-etc-kubernetes\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079808 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d084e88e-cb6f-4344-9d93-fee760d1d9c3-system-cni-dir\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079805 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-log-socket\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079825 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee61197e-cccc-4272-b95b-769b44f10298-tmp\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079844 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d084e88e-cb6f-4344-9d93-fee760d1d9c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079864 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d084e88e-cb6f-4344-9d93-fee760d1d9c3-system-cni-dir\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079870 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-etc-openvswitch\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079890 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-cni-netd\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.079933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079921 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3112e5e6-80ab-4355-b4cc-669776e66483-iptables-alerter-script\") pod \"iptables-alerter-hspnc\" (UID: \"3112e5e6-80ab-4355-b4cc-669776e66483\") " pod="openshift-network-operator/iptables-alerter-hspnc" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079969 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-cni-netd\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.079967 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-etc-openvswitch\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080035 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs\") pod \"network-metrics-daemon-cr5qg\" (UID: \"25941c59-99b0-40e3-98fd-687a15e8da9e\") " pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080067 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z97xj\" (UniqueName: \"kubernetes.io/projected/25941c59-99b0-40e3-98fd-687a15e8da9e-kube-api-access-z97xj\") pod \"network-metrics-daemon-cr5qg\" (UID: \"25941c59-99b0-40e3-98fd-687a15e8da9e\") " pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080098 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-run-openvswitch\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080140 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-node-log\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080166 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-multus-conf-dir\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080198 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-registration-dir\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080230 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bw6c\" (UniqueName: \"kubernetes.io/projected/d084e88e-cb6f-4344-9d93-fee760d1d9c3-kube-api-access-9bw6c\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080254 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a8abea8-0e83-46f9-9fc1-f22428d89987-tmp-dir\") pod \"node-resolver-klp4q\" (UID: \"0a8abea8-0e83-46f9-9fc1-f22428d89987\") " pod="openshift-dns/node-resolver-klp4q" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080272 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-kubelet\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080300 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080318 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080342 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-run-multus-certs\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080360 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080377 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-lib-modules\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.080476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080398 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02d39fbc-ea0c-43d7-a221-f825998aa6d8-ovnkube-config\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080448 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-sys\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080472 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6vz5\" (UniqueName: \"kubernetes.io/projected/ee61197e-cccc-4272-b95b-769b44f10298-kube-api-access-q6vz5\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080491 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-run-ovn\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080511 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-cni-bin\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080530 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-system-cni-dir\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080550 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d084e88e-cb6f-4344-9d93-fee760d1d9c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080547 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-cnibin\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080595 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080615 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-systemd-units\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080633 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnfls\" (UniqueName: \"kubernetes.io/projected/02d39fbc-ea0c-43d7-a221-f825998aa6d8-kube-api-access-nnfls\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080651 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1661ba5-6bd9-484b-97a0-8b28cb87e456-kubelet-config\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080671 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-multus-socket-dir-parent\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080687 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4cada06-9af5-4483-a7a4-eca67319ff23-host\") pod \"node-ca-zs6xt\" (UID: \"c4cada06-9af5-4483-a7a4-eca67319ff23\") " pod="openshift-image-registry/node-ca-zs6xt" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080691 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-run-openvswitch\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080704 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-run\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080704 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a8abea8-0e83-46f9-9fc1-f22428d89987-tmp-dir\") pod \"node-resolver-klp4q\" (UID: \"0a8abea8-0e83-46f9-9fc1-f22428d89987\") " pod="openshift-dns/node-resolver-klp4q" Jul 09 18:04:06.081142 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080721 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d084e88e-cb6f-4344-9d93-fee760d1d9c3-cnibin\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080753 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d084e88e-cb6f-4344-9d93-fee760d1d9c3-os-release\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080752 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-run-ovn\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080390 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d084e88e-cb6f-4344-9d93-fee760d1d9c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080648 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-node-log\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.080980 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-run-systemd\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081006 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-var-lib-openvswitch\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081029 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02d39fbc-ea0c-43d7-a221-f825998aa6d8-ovnkube-script-lib\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081048 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-multus-cni-dir\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081065 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-os-release\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081081 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq5xh\" (UniqueName: \"kubernetes.io/projected/e9c90246-157e-41aa-9498-a676922f8f0f-kube-api-access-qq5xh\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081091 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-systemd-units\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081103 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d084e88e-cb6f-4344-9d93-fee760d1d9c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081123 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d084e88e-cb6f-4344-9d93-fee760d1d9c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081310 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d084e88e-cb6f-4344-9d93-fee760d1d9c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081360 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-cni-bin\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081367 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d084e88e-cb6f-4344-9d93-fee760d1d9c3-os-release\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.082139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081383 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02d39fbc-ea0c-43d7-a221-f825998aa6d8-env-overrides\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081413 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-kubelet\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081419 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-socket-dir\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081448 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-sys-fs\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081477 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxpxr\" (UniqueName: \"kubernetes.io/projected/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-kube-api-access-gxpxr\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081482 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d084e88e-cb6f-4344-9d93-fee760d1d9c3-cnibin\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081447 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081531 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-var-lib-openvswitch\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081540 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c4cada06-9af5-4483-a7a4-eca67319ff23-serviceca\") pod \"node-ca-zs6xt\" (UID: \"c4cada06-9af5-4483-a7a4-eca67319ff23\") " pod="openshift-image-registry/node-ca-zs6xt" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081569 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-var-lib-kubelet\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081606 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee61197e-cccc-4272-b95b-769b44f10298-etc-tuned\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081644 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8s49\" (UniqueName: \"kubernetes.io/projected/0a8abea8-0e83-46f9-9fc1-f22428d89987-kube-api-access-f8s49\") pod \"node-resolver-klp4q\" (UID: \"0a8abea8-0e83-46f9-9fc1-f22428d89987\") " pod="openshift-dns/node-resolver-klp4q" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081677 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1661ba5-6bd9-484b-97a0-8b28cb87e456-dbus\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081701 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-var-lib-kubelet\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081735 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-device-dir\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081788 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-sysconfig\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081818 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-kubernetes\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.083006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081825 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02d39fbc-ea0c-43d7-a221-f825998aa6d8-env-overrides\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.083602 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081843 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-systemd\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.083602 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081873 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-host\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.083602 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081902 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-run-ovn-kubernetes\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.083602 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081919 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-run-systemd\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.083602 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.081931 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02d39fbc-ea0c-43d7-a221-f825998aa6d8-ovn-node-metrics-cert\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.083602 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.082199 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02d39fbc-ea0c-43d7-a221-f825998aa6d8-ovnkube-script-lib\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.083602 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.082286 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02d39fbc-ea0c-43d7-a221-f825998aa6d8-host-run-ovn-kubernetes\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.083602 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.082349 2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 18:04:06.083602 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.082396 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02d39fbc-ea0c-43d7-a221-f825998aa6d8-ovnkube-config\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.083602 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.082463 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d084e88e-cb6f-4344-9d93-fee760d1d9c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.083602 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.082888 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-run-netns\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.083602 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.082953 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-sysctl-conf\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.088969 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.088804 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02d39fbc-ea0c-43d7-a221-f825998aa6d8-ovn-node-metrics-cert\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.097819 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.097787 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bw6c\" (UniqueName: \"kubernetes.io/projected/d084e88e-cb6f-4344-9d93-fee760d1d9c3-kube-api-access-9bw6c\") pod \"multus-additional-cni-plugins-xvhp7\" (UID: \"d084e88e-cb6f-4344-9d93-fee760d1d9c3\") " pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.098128 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.098090 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8s49\" (UniqueName: \"kubernetes.io/projected/0a8abea8-0e83-46f9-9fc1-f22428d89987-kube-api-access-f8s49\") pod \"node-resolver-klp4q\" (UID: \"0a8abea8-0e83-46f9-9fc1-f22428d89987\") " pod="openshift-dns/node-resolver-klp4q" Jul 09 18:04:06.098452 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.098426 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnfls\" (UniqueName: \"kubernetes.io/projected/02d39fbc-ea0c-43d7-a221-f825998aa6d8-kube-api-access-nnfls\") pod \"ovnkube-node-sjxbv\" (UID: \"02d39fbc-ea0c-43d7-a221-f825998aa6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.183597 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183514 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-multus-cni-dir\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.183597 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183546 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-os-release\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.183597 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183572 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq5xh\" (UniqueName: \"kubernetes.io/projected/e9c90246-157e-41aa-9498-a676922f8f0f-kube-api-access-qq5xh\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.183888 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183601 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-socket-dir\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.183888 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183623 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-sys-fs\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.183888 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183641 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxpxr\" (UniqueName: \"kubernetes.io/projected/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-kube-api-access-gxpxr\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.183888 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183657 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c4cada06-9af5-4483-a7a4-eca67319ff23-serviceca\") pod \"node-ca-zs6xt\" (UID: \"c4cada06-9af5-4483-a7a4-eca67319ff23\") " pod="openshift-image-registry/node-ca-zs6xt" Jul 09 18:04:06.183888 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183672 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-var-lib-kubelet\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.183888 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183687 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee61197e-cccc-4272-b95b-769b44f10298-etc-tuned\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.183888 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183702 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1661ba5-6bd9-484b-97a0-8b28cb87e456-dbus\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:06.183888 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183736 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-var-lib-kubelet\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.183888 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183753 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-device-dir\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.183888 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183768 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-sysconfig\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.183888 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183805 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-kubernetes\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.183888 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183828 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-systemd\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.183888 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183854 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-host\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.183888 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183880 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-run-netns\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183904 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-sysctl-conf\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183923 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq4d7\" (UniqueName: \"kubernetes.io/projected/3112e5e6-80ab-4355-b4cc-669776e66483-kube-api-access-rq4d7\") pod \"iptables-alerter-hspnc\" (UID: \"3112e5e6-80ab-4355-b4cc-669776e66483\") " pod="openshift-network-operator/iptables-alerter-hspnc" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183938 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjffd\" (UniqueName: \"kubernetes.io/projected/c4cada06-9af5-4483-a7a4-eca67319ff23-kube-api-access-pjffd\") pod \"node-ca-zs6xt\" (UID: \"c4cada06-9af5-4483-a7a4-eca67319ff23\") " pod="openshift-image-registry/node-ca-zs6xt" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183955 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qqrb\" (UniqueName: \"kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb\") pod \"network-check-target-lntgg\" (UID: \"aa5fad78-82ce-49bb-a50e-2b178586bdaf\") " pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.183975 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3fa7022d-512b-4625-95bb-1342a17d197f-agent-certs\") pod \"konnectivity-agent-2qp88\" (UID: \"3fa7022d-512b-4625-95bb-1342a17d197f\") " pod="kube-system/konnectivity-agent-2qp88" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184000 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-var-lib-cni-bin\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184021 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-hostroot\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184051 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3fa7022d-512b-4625-95bb-1342a17d197f-konnectivity-ca\") pod \"konnectivity-agent-2qp88\" (UID: \"3fa7022d-512b-4625-95bb-1342a17d197f\") " pod="kube-system/konnectivity-agent-2qp88" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184067 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-run-k8s-cni-cncf-io\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184091 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-var-lib-cni-multus\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184119 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3112e5e6-80ab-4355-b4cc-669776e66483-host-slash\") pod \"iptables-alerter-hspnc\" (UID: \"3112e5e6-80ab-4355-b4cc-669776e66483\") " pod="openshift-network-operator/iptables-alerter-hspnc" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184145 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-modprobe-d\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184171 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-sysctl-d\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184196 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9c90246-157e-41aa-9498-a676922f8f0f-cni-binary-copy\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184219 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9c90246-157e-41aa-9498-a676922f8f0f-multus-daemon-config\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184249 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-etc-kubernetes\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.184418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184279 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee61197e-cccc-4272-b95b-769b44f10298-tmp\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184316 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3112e5e6-80ab-4355-b4cc-669776e66483-iptables-alerter-script\") pod \"iptables-alerter-hspnc\" (UID: \"3112e5e6-80ab-4355-b4cc-669776e66483\") " pod="openshift-network-operator/iptables-alerter-hspnc" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184333 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs\") pod \"network-metrics-daemon-cr5qg\" (UID: \"25941c59-99b0-40e3-98fd-687a15e8da9e\") " pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184349 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z97xj\" (UniqueName: \"kubernetes.io/projected/25941c59-99b0-40e3-98fd-687a15e8da9e-kube-api-access-z97xj\") pod \"network-metrics-daemon-cr5qg\" (UID: \"25941c59-99b0-40e3-98fd-687a15e8da9e\") " pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184369 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-multus-conf-dir\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184390 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-registration-dir\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184412 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184427 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-run-multus-certs\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184445 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184474 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-lib-modules\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184498 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-sys\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184524 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6vz5\" (UniqueName: \"kubernetes.io/projected/ee61197e-cccc-4272-b95b-769b44f10298-kube-api-access-q6vz5\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184541 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-system-cni-dir\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184558 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-cnibin\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184574 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184599 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1661ba5-6bd9-484b-97a0-8b28cb87e456-kubelet-config\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184625 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-multus-socket-dir-parent\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.185095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184642 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4cada06-9af5-4483-a7a4-eca67319ff23-host\") pod \"node-ca-zs6xt\" (UID: \"c4cada06-9af5-4483-a7a4-eca67319ff23\") " pod="openshift-image-registry/node-ca-zs6xt" Jul 09 18:04:06.185731 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184663 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-run\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.185731 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184777 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-run\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.185731 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184852 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-multus-cni-dir\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.185731 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.184955 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-os-release\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.185731 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.185278 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-socket-dir\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.185731 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.185328 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-sys-fs\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.185990 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.185845 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c4cada06-9af5-4483-a7a4-eca67319ff23-serviceca\") pod \"node-ca-zs6xt\" (UID: \"c4cada06-9af5-4483-a7a4-eca67319ff23\") " pod="openshift-image-registry/node-ca-zs6xt" Jul 09 18:04:06.185990 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.185908 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-var-lib-kubelet\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.186290 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186260 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-sysctl-d\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.186382 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186357 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1661ba5-6bd9-484b-97a0-8b28cb87e456-dbus\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:06.186426 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186386 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-var-lib-kubelet\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.186470 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186421 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-device-dir\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.186470 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186455 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-sysconfig\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.186542 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186491 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-kubernetes\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.186542 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186536 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-systemd\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.186614 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186569 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-host\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.186614 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186595 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-run-netns\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.186679 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186668 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-sysctl-conf\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186801 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-hostroot\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186852 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-var-lib-cni-multus\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186890 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-var-lib-cni-bin\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186920 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-run-k8s-cni-cncf-io\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186939 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-host-run-multus-certs\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.186941 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-etc-modprobe-d\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187005 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187095 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3112e5e6-80ab-4355-b4cc-669776e66483-host-slash\") pod \"iptables-alerter-hspnc\" (UID: \"3112e5e6-80ab-4355-b4cc-669776e66483\") " pod="openshift-network-operator/iptables-alerter-hspnc" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187116 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-multus-conf-dir\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187177 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-registration-dir\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187177 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1661ba5-6bd9-484b-97a0-8b28cb87e456-kubelet-config\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187229 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-cnibin\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187271 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3fa7022d-512b-4625-95bb-1342a17d197f-konnectivity-ca\") pod \"konnectivity-agent-2qp88\" (UID: \"3fa7022d-512b-4625-95bb-1342a17d197f\") " pod="kube-system/konnectivity-agent-2qp88" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187288 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.187275 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187312 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-system-cni-dir\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.187863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187349 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-multus-socket-dir-parent\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.188742 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.187358 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret podName:e1661ba5-6bd9-484b-97a0-8b28cb87e456 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:06.687342394 +0000 UTC m=+3.152864785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret") pod "global-pull-secret-syncer-sq92v" (UID: "e1661ba5-6bd9-484b-97a0-8b28cb87e456") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:04:06.188742 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187397 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9c90246-157e-41aa-9498-a676922f8f0f-etc-kubernetes\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.188742 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187520 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4cada06-9af5-4483-a7a4-eca67319ff23-host\") pod \"node-ca-zs6xt\" (UID: \"c4cada06-9af5-4483-a7a4-eca67319ff23\") " pod="openshift-image-registry/node-ca-zs6xt" Jul 09 18:04:06.188742 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187562 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-sys\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.188742 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187574 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3112e5e6-80ab-4355-b4cc-669776e66483-iptables-alerter-script\") pod \"iptables-alerter-hspnc\" (UID: \"3112e5e6-80ab-4355-b4cc-669776e66483\") " pod="openshift-network-operator/iptables-alerter-hspnc" Jul 09 18:04:06.188742 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187625 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee61197e-cccc-4272-b95b-769b44f10298-lib-modules\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.188742 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187784 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9c90246-157e-41aa-9498-a676922f8f0f-cni-binary-copy\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.188742 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.187864 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:04:06.188742 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.187898 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9c90246-157e-41aa-9498-a676922f8f0f-multus-daemon-config\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.188742 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.187967 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs podName:25941c59-99b0-40e3-98fd-687a15e8da9e nodeName:}" failed. No retries permitted until 2026-07-09 18:04:06.687909961 +0000 UTC m=+3.153432351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs") pod "network-metrics-daemon-cr5qg" (UID: "25941c59-99b0-40e3-98fd-687a15e8da9e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:04:06.190217 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.190193 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3fa7022d-512b-4625-95bb-1342a17d197f-agent-certs\") pod \"konnectivity-agent-2qp88\" (UID: \"3fa7022d-512b-4625-95bb-1342a17d197f\") " pod="kube-system/konnectivity-agent-2qp88" Jul 09 18:04:06.190310 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.190281 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee61197e-cccc-4272-b95b-769b44f10298-tmp\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.192269 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.191956 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee61197e-cccc-4272-b95b-769b44f10298-etc-tuned\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.196804 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.196755 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" event={"ID":"631442f90a9aec34b0d279ea0f5f9c73","Type":"ContainerStarted","Data":"89555452aed2a17ed2074c814dbaabc039c27bb7c9284a413f7fd6e780b523b4"} Jul 09 18:04:06.198637 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.198610 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" event={"ID":"6fca482dcad147ae6c62a690d3cba064","Type":"ContainerStarted","Data":"a956048cb7c2789aba0b4fabcebd98412c6f4073274fa3c4f2479228a556f694"} Jul 09 18:04:06.202299 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.202276 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxpxr\" (UniqueName: \"kubernetes.io/projected/b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8-kube-api-access-gxpxr\") pod \"aws-ebs-csi-driver-node-4h27j\" (UID: \"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.202659 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.202640 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq5xh\" (UniqueName: \"kubernetes.io/projected/e9c90246-157e-41aa-9498-a676922f8f0f-kube-api-access-qq5xh\") pod \"multus-tkxzm\" (UID: \"e9c90246-157e-41aa-9498-a676922f8f0f\") " pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.206102 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.206081 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:04:06.206312 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.206106 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:04:06.206312 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.206123 2569 projected.go:196] Error preparing data for projected volume kube-api-access-4qqrb for pod openshift-network-diagnostics/network-check-target-lntgg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:04:06.206312 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.206195 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb podName:aa5fad78-82ce-49bb-a50e-2b178586bdaf nodeName:}" failed. No retries permitted until 2026-07-09 18:04:06.706175475 +0000 UTC m=+3.171697891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4qqrb" (UniqueName: "kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb") pod "network-check-target-lntgg" (UID: "aa5fad78-82ce-49bb-a50e-2b178586bdaf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:04:06.209718 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.209691 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjffd\" (UniqueName: \"kubernetes.io/projected/c4cada06-9af5-4483-a7a4-eca67319ff23-kube-api-access-pjffd\") pod \"node-ca-zs6xt\" (UID: \"c4cada06-9af5-4483-a7a4-eca67319ff23\") " pod="openshift-image-registry/node-ca-zs6xt" Jul 09 18:04:06.209832 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.209797 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq4d7\" (UniqueName: \"kubernetes.io/projected/3112e5e6-80ab-4355-b4cc-669776e66483-kube-api-access-rq4d7\") pod \"iptables-alerter-hspnc\" (UID: \"3112e5e6-80ab-4355-b4cc-669776e66483\") " pod="openshift-network-operator/iptables-alerter-hspnc" Jul 09 18:04:06.211108 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.211081 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6vz5\" (UniqueName: \"kubernetes.io/projected/ee61197e-cccc-4272-b95b-769b44f10298-kube-api-access-q6vz5\") pod \"tuned-kwgh9\" (UID: \"ee61197e-cccc-4272-b95b-769b44f10298\") " pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.211959 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.211939 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z97xj\" (UniqueName: \"kubernetes.io/projected/25941c59-99b0-40e3-98fd-687a15e8da9e-kube-api-access-z97xj\") pod \"network-metrics-daemon-cr5qg\" (UID: \"25941c59-99b0-40e3-98fd-687a15e8da9e\") " pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:06.277829 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.277797 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xvhp7" Jul 09 18:04:06.295809 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.295777 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-klp4q" Jul 09 18:04:06.305911 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.305891 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:06.313312 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.313289 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2qp88" Jul 09 18:04:06.322308 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.322281 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" Jul 09 18:04:06.331150 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.331134 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zs6xt" Jul 09 18:04:06.339932 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.339907 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hspnc" Jul 09 18:04:06.348824 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.348798 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tkxzm" Jul 09 18:04:06.355706 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.355666 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" Jul 09 18:04:06.374359 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.374328 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:04:06.689338 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.689279 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs\") pod \"network-metrics-daemon-cr5qg\" (UID: \"25941c59-99b0-40e3-98fd-687a15e8da9e\") " pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:06.689338 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.689340 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:06.689601 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.689469 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:04:06.689601 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.689501 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:04:06.689601 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.689557 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret podName:e1661ba5-6bd9-484b-97a0-8b28cb87e456 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:07.689536433 +0000 UTC m=+4.155058838 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret") pod "global-pull-secret-syncer-sq92v" (UID: "e1661ba5-6bd9-484b-97a0-8b28cb87e456") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:04:06.689601 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.689577 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs podName:25941c59-99b0-40e3-98fd-687a15e8da9e nodeName:}" failed. No retries permitted until 2026-07-09 18:04:07.689565982 +0000 UTC m=+4.155088375 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs") pod "network-metrics-daemon-cr5qg" (UID: "25941c59-99b0-40e3-98fd-687a15e8da9e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:04:06.789943 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:06.789908 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qqrb\" (UniqueName: \"kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb\") pod \"network-check-target-lntgg\" (UID: \"aa5fad78-82ce-49bb-a50e-2b178586bdaf\") " pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:06.790136 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.790116 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:04:06.790210 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.790142 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:04:06.790210 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.790157 2569 projected.go:196] Error preparing data for projected volume kube-api-access-4qqrb for pod openshift-network-diagnostics/network-check-target-lntgg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:04:06.790316 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:06.790226 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb podName:aa5fad78-82ce-49bb-a50e-2b178586bdaf nodeName:}" failed. No retries permitted until 2026-07-09 18:04:07.790205814 +0000 UTC m=+4.255728222 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4qqrb" (UniqueName: "kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb") pod "network-check-target-lntgg" (UID: "aa5fad78-82ce-49bb-a50e-2b178586bdaf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:04:07.068093 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:07.068036 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 17:59:06 +0000 UTC" deadline="2028-01-11 20:13:06.679510674 +0000 UTC" Jul 09 18:04:07.068093 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:07.068083 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13226h8m59.611430157s" Jul 09 18:04:07.191423 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:07.191384 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:07.191609 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:07.191521 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:07.222088 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:07.222060 2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 18:04:07.290069 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:07.290033 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7a7bd06_ebf8_43f4_acf5_7a62b07b73c8.slice/crio-f223fb3ed1ec6ac73fdf7e4ae29d6eac947c587ac118b2c00a19e196b716dee5 WatchSource:0}: Error finding container f223fb3ed1ec6ac73fdf7e4ae29d6eac947c587ac118b2c00a19e196b716dee5: Status 404 returned error can't find the container with id f223fb3ed1ec6ac73fdf7e4ae29d6eac947c587ac118b2c00a19e196b716dee5 Jul 09 18:04:07.290809 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:07.290781 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa7022d_512b_4625_95bb_1342a17d197f.slice/crio-89928c8c06200870a4039232f5fd16d3cf2c8058332ab00c8a2a1a79c436f535 WatchSource:0}: Error finding container 89928c8c06200870a4039232f5fd16d3cf2c8058332ab00c8a2a1a79c436f535: Status 404 returned error can't find the container with id 89928c8c06200870a4039232f5fd16d3cf2c8058332ab00c8a2a1a79c436f535 Jul 09 18:04:07.291910 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:07.291884 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3112e5e6_80ab_4355_b4cc_669776e66483.slice/crio-92ec76d24ecc4da72e55026491cfa070a465afcc843a137608c03f62b2252366 WatchSource:0}: Error finding container 92ec76d24ecc4da72e55026491cfa070a465afcc843a137608c03f62b2252366: Status 404 returned error can't find the container with id 92ec76d24ecc4da72e55026491cfa070a465afcc843a137608c03f62b2252366 Jul 09 18:04:07.293153 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:07.293067 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a8abea8_0e83_46f9_9fc1_f22428d89987.slice/crio-8f99cc4f1c6ad110ea7942fbdef1cfdcf1d687ed4be78c5717b5981972e562cc WatchSource:0}: Error finding container 8f99cc4f1c6ad110ea7942fbdef1cfdcf1d687ed4be78c5717b5981972e562cc: Status 404 returned error can't find the container with id 8f99cc4f1c6ad110ea7942fbdef1cfdcf1d687ed4be78c5717b5981972e562cc Jul 09 18:04:07.294669 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:07.294643 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4cada06_9af5_4483_a7a4_eca67319ff23.slice/crio-086571c3c6bf23e5c978495bc330aa65814c8d84e454bc2e74dffde0af65d14d WatchSource:0}: Error finding container 086571c3c6bf23e5c978495bc330aa65814c8d84e454bc2e74dffde0af65d14d: Status 404 returned error can't find the container with id 086571c3c6bf23e5c978495bc330aa65814c8d84e454bc2e74dffde0af65d14d Jul 09 18:04:07.295732 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:07.295673 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02d39fbc_ea0c_43d7_a221_f825998aa6d8.slice/crio-8fde711a60fa38be9a6fabf69cc333f1db296fde2c826395b900b7c077e0426f WatchSource:0}: Error finding container 8fde711a60fa38be9a6fabf69cc333f1db296fde2c826395b900b7c077e0426f: Status 404 returned error can't find the container with id 8fde711a60fa38be9a6fabf69cc333f1db296fde2c826395b900b7c077e0426f Jul 09 18:04:07.299162 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:07.299146 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd084e88e_cb6f_4344_9d93_fee760d1d9c3.slice/crio-5444c565d3b181e79b9d269888bca7e3e56f24c54d3aa69cb8b87f664f3ad2a4 WatchSource:0}: Error finding container 5444c565d3b181e79b9d269888bca7e3e56f24c54d3aa69cb8b87f664f3ad2a4: Status 404 returned error can't find the container with id 5444c565d3b181e79b9d269888bca7e3e56f24c54d3aa69cb8b87f664f3ad2a4 Jul 09 18:04:07.303066 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:07.302861 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee61197e_cccc_4272_b95b_769b44f10298.slice/crio-a810558256ef52db922512d3de5dbee8a6e6a91faeb163ee2fc45718f487327f WatchSource:0}: Error finding container a810558256ef52db922512d3de5dbee8a6e6a91faeb163ee2fc45718f487327f: Status 404 returned error can't find the container with id a810558256ef52db922512d3de5dbee8a6e6a91faeb163ee2fc45718f487327f Jul 09 18:04:07.305068 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:07.304725 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c90246_157e_41aa_9498_a676922f8f0f.slice/crio-b8f7bd65ddd9561f1c8bbe1fe118dbcd94ca86344fa8f9f57924e4e9fce45641 WatchSource:0}: Error finding container b8f7bd65ddd9561f1c8bbe1fe118dbcd94ca86344fa8f9f57924e4e9fce45641: Status 404 returned error can't find the container with id b8f7bd65ddd9561f1c8bbe1fe118dbcd94ca86344fa8f9f57924e4e9fce45641 Jul 09 18:04:07.696757 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:07.696654 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs\") pod \"network-metrics-daemon-cr5qg\" (UID: \"25941c59-99b0-40e3-98fd-687a15e8da9e\") " pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:07.696757 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:07.696740 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:07.696985 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:07.696838 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:04:07.696985 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:07.696918 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs podName:25941c59-99b0-40e3-98fd-687a15e8da9e nodeName:}" failed. No retries permitted until 2026-07-09 18:04:09.696897994 +0000 UTC m=+6.162420400 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs") pod "network-metrics-daemon-cr5qg" (UID: "25941c59-99b0-40e3-98fd-687a15e8da9e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:04:07.696985 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:07.696839 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:04:07.696985 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:07.696983 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret podName:e1661ba5-6bd9-484b-97a0-8b28cb87e456 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:09.696968545 +0000 UTC m=+6.162490940 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret") pod "global-pull-secret-syncer-sq92v" (UID: "e1661ba5-6bd9-484b-97a0-8b28cb87e456") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:04:07.797431 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:07.797390 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qqrb\" (UniqueName: \"kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb\") pod \"network-check-target-lntgg\" (UID: \"aa5fad78-82ce-49bb-a50e-2b178586bdaf\") " pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:07.797653 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:07.797618 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:04:07.797762 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:07.797655 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:04:07.797762 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:07.797673 2569 projected.go:196] Error preparing data for projected volume kube-api-access-4qqrb for pod openshift-network-diagnostics/network-check-target-lntgg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:04:07.797762 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:07.797756 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb podName:aa5fad78-82ce-49bb-a50e-2b178586bdaf nodeName:}" failed. No retries permitted until 2026-07-09 18:04:09.797736717 +0000 UTC m=+6.263259127 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4qqrb" (UniqueName: "kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb") pod "network-check-target-lntgg" (UID: "aa5fad78-82ce-49bb-a50e-2b178586bdaf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:04:08.068782 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.068681 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 17:59:06 +0000 UTC" deadline="2028-02-24 20:20:25.189113014 +0000 UTC" Jul 09 18:04:08.068782 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.068744 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14282h16m17.120374484s" Jul 09 18:04:08.194187 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.193455 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:08.194187 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:08.193600 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:08.194187 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.194045 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:08.194187 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:08.194124 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:08.219419 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.219378 2569 generic.go:356] "Generic (PLEG): container finished" podID="6fca482dcad147ae6c62a690d3cba064" containerID="9cc445aa043a9acb816b61e0e8e324cd3a52aa9b8df64a1933cc9365443b24f0" exitCode=0 Jul 09 18:04:08.219596 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.219493 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" event={"ID":"6fca482dcad147ae6c62a690d3cba064","Type":"ContainerDied","Data":"9cc445aa043a9acb816b61e0e8e324cd3a52aa9b8df64a1933cc9365443b24f0"} Jul 09 18:04:08.226705 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.226671 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tkxzm" event={"ID":"e9c90246-157e-41aa-9498-a676922f8f0f","Type":"ContainerStarted","Data":"b8f7bd65ddd9561f1c8bbe1fe118dbcd94ca86344fa8f9f57924e4e9fce45641"} Jul 09 18:04:08.239863 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.239815 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhp7" event={"ID":"d084e88e-cb6f-4344-9d93-fee760d1d9c3","Type":"ContainerStarted","Data":"5444c565d3b181e79b9d269888bca7e3e56f24c54d3aa69cb8b87f664f3ad2a4"} Jul 09 18:04:08.248101 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.248044 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" event={"ID":"ee61197e-cccc-4272-b95b-769b44f10298","Type":"ContainerStarted","Data":"a810558256ef52db922512d3de5dbee8a6e6a91faeb163ee2fc45718f487327f"} Jul 09 18:04:08.256569 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.256434 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" event={"ID":"02d39fbc-ea0c-43d7-a221-f825998aa6d8","Type":"ContainerStarted","Data":"8fde711a60fa38be9a6fabf69cc333f1db296fde2c826395b900b7c077e0426f"} Jul 09 18:04:08.259236 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.259199 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hspnc" event={"ID":"3112e5e6-80ab-4355-b4cc-669776e66483","Type":"ContainerStarted","Data":"92ec76d24ecc4da72e55026491cfa070a465afcc843a137608c03f62b2252366"} Jul 09 18:04:08.262186 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.262097 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" event={"ID":"631442f90a9aec34b0d279ea0f5f9c73","Type":"ContainerStarted","Data":"c079c7894f004d19e82cf6903caa21837df40d82e3deb1a8b250e344ad3c0527"} Jul 09 18:04:08.268706 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.268642 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zs6xt" event={"ID":"c4cada06-9af5-4483-a7a4-eca67319ff23","Type":"ContainerStarted","Data":"086571c3c6bf23e5c978495bc330aa65814c8d84e454bc2e74dffde0af65d14d"} Jul 09 18:04:08.274914 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.274885 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-klp4q" event={"ID":"0a8abea8-0e83-46f9-9fc1-f22428d89987","Type":"ContainerStarted","Data":"8f99cc4f1c6ad110ea7942fbdef1cfdcf1d687ed4be78c5717b5981972e562cc"} Jul 09 18:04:08.280376 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.280346 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2qp88" event={"ID":"3fa7022d-512b-4625-95bb-1342a17d197f","Type":"ContainerStarted","Data":"89928c8c06200870a4039232f5fd16d3cf2c8058332ab00c8a2a1a79c436f535"} Jul 09 18:04:08.288611 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:08.288181 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" event={"ID":"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8","Type":"ContainerStarted","Data":"f223fb3ed1ec6ac73fdf7e4ae29d6eac947c587ac118b2c00a19e196b716dee5"} Jul 09 18:04:09.191452 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:09.191363 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:09.192007 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:09.191519 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:09.319538 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:09.319455 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" event={"ID":"6fca482dcad147ae6c62a690d3cba064","Type":"ContainerStarted","Data":"f24a07338d7a6d5e936617611eda62d60a903f63937b627fe85c0c83f3bf3ea8"} Jul 09 18:04:09.342510 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:09.341893 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-168.ec2.internal" podStartSLOduration=4.341871759 podStartE2EDuration="4.341871759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:04:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:08.280331516 +0000 UTC m=+4.745853931" watchObservedRunningTime="2026-07-09 18:04:09.341871759 +0000 UTC m=+5.807394178" Jul 09 18:04:09.714217 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:09.713529 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs\") pod \"network-metrics-daemon-cr5qg\" (UID: \"25941c59-99b0-40e3-98fd-687a15e8da9e\") " pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:09.714217 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:09.713583 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:09.714217 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:09.713728 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:04:09.714217 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:09.713792 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret podName:e1661ba5-6bd9-484b-97a0-8b28cb87e456 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:13.713773909 +0000 UTC m=+10.179296300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret") pod "global-pull-secret-syncer-sq92v" (UID: "e1661ba5-6bd9-484b-97a0-8b28cb87e456") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:04:09.714217 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:09.713859 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:04:09.714217 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:09.713892 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs podName:25941c59-99b0-40e3-98fd-687a15e8da9e nodeName:}" failed. No retries permitted until 2026-07-09 18:04:13.7138802 +0000 UTC m=+10.179402591 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs") pod "network-metrics-daemon-cr5qg" (UID: "25941c59-99b0-40e3-98fd-687a15e8da9e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:04:09.815100 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:09.814468 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qqrb\" (UniqueName: \"kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb\") pod \"network-check-target-lntgg\" (UID: \"aa5fad78-82ce-49bb-a50e-2b178586bdaf\") " pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:09.815100 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:09.814650 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:04:09.815100 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:09.814669 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:04:09.815100 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:09.814681 2569 projected.go:196] Error preparing data for projected volume kube-api-access-4qqrb for pod openshift-network-diagnostics/network-check-target-lntgg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:04:09.815100 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:09.814761 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb podName:aa5fad78-82ce-49bb-a50e-2b178586bdaf nodeName:}" failed. No retries permitted until 2026-07-09 18:04:13.814744143 +0000 UTC m=+10.280266547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4qqrb" (UniqueName: "kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb") pod "network-check-target-lntgg" (UID: "aa5fad78-82ce-49bb-a50e-2b178586bdaf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:04:10.191711 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:10.191049 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:10.191711 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:10.191185 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:10.191711 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:10.191532 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:10.191711 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:10.191612 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:11.191228 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:11.191196 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:11.191424 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:11.191340 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:12.195018 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:12.194989 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:12.195388 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:12.195105 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:12.195523 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:12.195506 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:12.195624 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:12.195598 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:13.191666 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:13.191628 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:13.191874 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:13.191766 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:13.748106 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:13.748071 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs\") pod \"network-metrics-daemon-cr5qg\" (UID: \"25941c59-99b0-40e3-98fd-687a15e8da9e\") " pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:13.748566 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:13.748116 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:13.748566 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:13.748260 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:04:13.748566 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:13.748321 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret podName:e1661ba5-6bd9-484b-97a0-8b28cb87e456 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:21.748302265 +0000 UTC m=+18.213824654 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret") pod "global-pull-secret-syncer-sq92v" (UID: "e1661ba5-6bd9-484b-97a0-8b28cb87e456") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:04:13.748759 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:13.748733 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:04:13.748807 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:13.748790 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs podName:25941c59-99b0-40e3-98fd-687a15e8da9e nodeName:}" failed. No retries permitted until 2026-07-09 18:04:21.748773272 +0000 UTC m=+18.214295664 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs") pod "network-metrics-daemon-cr5qg" (UID: "25941c59-99b0-40e3-98fd-687a15e8da9e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:04:13.848984 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:13.848944 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qqrb\" (UniqueName: \"kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb\") pod \"network-check-target-lntgg\" (UID: \"aa5fad78-82ce-49bb-a50e-2b178586bdaf\") " pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:13.849150 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:13.849103 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:04:13.849150 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:13.849121 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:04:13.849150 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:13.849131 2569 projected.go:196] Error preparing data for projected volume kube-api-access-4qqrb for pod openshift-network-diagnostics/network-check-target-lntgg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:04:13.849310 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:13.849185 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb podName:aa5fad78-82ce-49bb-a50e-2b178586bdaf nodeName:}" failed. No retries permitted until 2026-07-09 18:04:21.849168994 +0000 UTC m=+18.314691388 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4qqrb" (UniqueName: "kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb") pod "network-check-target-lntgg" (UID: "aa5fad78-82ce-49bb-a50e-2b178586bdaf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:04:14.191277 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:14.191242 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:14.191427 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:14.191375 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:14.193729 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:14.193573 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:14.193729 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:14.193667 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:15.191044 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:15.191017 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:15.191426 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:15.191144 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:16.195020 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:16.194582 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:16.195020 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:16.194712 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:16.195514 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:16.195192 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:16.195514 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:16.195315 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:17.191893 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:17.191864 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:17.192060 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:17.191996 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:18.191974 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:18.191929 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:18.192416 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:18.192071 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:18.192416 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:18.192084 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:18.192416 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:18.192195 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:19.191089 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:19.191053 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:19.191267 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:19.191180 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:20.190954 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:20.190919 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:20.191413 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:20.190919 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:20.191413 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:20.191041 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:20.191413 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:20.191125 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:21.191422 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:21.191386 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:21.191925 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:21.191521 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:21.804571 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:21.804009 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs\") pod \"network-metrics-daemon-cr5qg\" (UID: \"25941c59-99b0-40e3-98fd-687a15e8da9e\") " pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:21.804571 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:21.804067 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:21.804571 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:21.804181 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:04:21.804571 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:21.804181 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:04:21.804571 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:21.804248 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret podName:e1661ba5-6bd9-484b-97a0-8b28cb87e456 nodeName:}" failed. No retries permitted until 2026-07-09 18:04:37.804231531 +0000 UTC m=+34.269753921 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret") pod "global-pull-secret-syncer-sq92v" (UID: "e1661ba5-6bd9-484b-97a0-8b28cb87e456") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:04:21.804571 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:21.804265 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs podName:25941c59-99b0-40e3-98fd-687a15e8da9e nodeName:}" failed. No retries permitted until 2026-07-09 18:04:37.804256356 +0000 UTC m=+34.269778745 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs") pod "network-metrics-daemon-cr5qg" (UID: "25941c59-99b0-40e3-98fd-687a15e8da9e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:04:21.905220 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:21.905187 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qqrb\" (UniqueName: \"kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb\") pod \"network-check-target-lntgg\" (UID: \"aa5fad78-82ce-49bb-a50e-2b178586bdaf\") " pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:21.905451 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:21.905379 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:04:21.905451 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:21.905405 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:04:21.905451 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:21.905419 2569 projected.go:196] Error preparing data for projected volume kube-api-access-4qqrb for pod openshift-network-diagnostics/network-check-target-lntgg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:04:21.905582 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:21.905485 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb podName:aa5fad78-82ce-49bb-a50e-2b178586bdaf nodeName:}" failed. No retries permitted until 2026-07-09 18:04:37.905464702 +0000 UTC m=+34.370987115 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4qqrb" (UniqueName: "kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb") pod "network-check-target-lntgg" (UID: "aa5fad78-82ce-49bb-a50e-2b178586bdaf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:04:22.191367 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:22.191283 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:22.191367 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:22.191317 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:22.191859 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:22.191408 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:22.191859 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:22.191529 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:23.190996 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:23.190966 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:23.191170 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:23.191080 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:24.192987 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:24.192309 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:24.192987 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:24.192374 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:24.192987 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:24.192619 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:24.192987 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:24.192713 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:25.191219 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:25.191181 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:25.191468 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:25.191340 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:26.191846 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:26.191810 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:26.192384 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:26.191935 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:26.192384 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:26.191995 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:26.192384 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:26.192128 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:27.191652 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.191494 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:27.191777 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:27.191758 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:27.364307 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.364278 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zs6xt" event={"ID":"c4cada06-9af5-4483-a7a4-eca67319ff23","Type":"ContainerStarted","Data":"19d7b707b2b2757f6a9cc3b83bfcb9cc056ceca4c4e714cd61b2d5034ca31bb7"} Jul 09 18:04:27.365823 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.365789 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-klp4q" event={"ID":"0a8abea8-0e83-46f9-9fc1-f22428d89987","Type":"ContainerStarted","Data":"b1f05531dc39913907710d2b1ed829b16d2e79e34a3b7e53133187a47037583d"} Jul 09 18:04:27.366941 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.366918 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2qp88" event={"ID":"3fa7022d-512b-4625-95bb-1342a17d197f","Type":"ContainerStarted","Data":"f4e21e82eb2f53c00d94215e7f4fd1f3b0327ecfa66313e5845c945bf1cc6d8e"} Jul 09 18:04:27.368232 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.368212 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" event={"ID":"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8","Type":"ContainerStarted","Data":"adca36866467de9e85399494cab76ddb0583158b486f409c1015bd7def7d06f1"} Jul 09 18:04:27.369426 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.369403 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tkxzm" event={"ID":"e9c90246-157e-41aa-9498-a676922f8f0f","Type":"ContainerStarted","Data":"eb71be69b59bc53c9fb90e0b94501564ed3e080779858f6e2e1b5b16f5999a5d"} Jul 09 18:04:27.370682 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.370663 2569 generic.go:356] "Generic (PLEG): container finished" podID="d084e88e-cb6f-4344-9d93-fee760d1d9c3" containerID="4b7b2d945474549e31a7e0d6343393f5d66ad50964c1592fbd9ddb8ceda07a4f" exitCode=0 Jul 09 18:04:27.370772 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.370714 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhp7" event={"ID":"d084e88e-cb6f-4344-9d93-fee760d1d9c3","Type":"ContainerDied","Data":"4b7b2d945474549e31a7e0d6343393f5d66ad50964c1592fbd9ddb8ceda07a4f"} Jul 09 18:04:27.372131 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.372107 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" event={"ID":"ee61197e-cccc-4272-b95b-769b44f10298","Type":"ContainerStarted","Data":"fc2d562f788ceeb5b0d38c2335fae4b30eced2be77c9016eb9eb2325be3fda86"} Jul 09 18:04:27.374668 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.374646 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" event={"ID":"02d39fbc-ea0c-43d7-a221-f825998aa6d8","Type":"ContainerStarted","Data":"3aeb05d5adf0f2273e293f83b697b4144e7a1983d234e9e74738f4bfe647f197"} Jul 09 18:04:27.374753 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.374741 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" event={"ID":"02d39fbc-ea0c-43d7-a221-f825998aa6d8","Type":"ContainerStarted","Data":"ddaa8fd00d206869a996586eeb34077c22ef8a9bba095195b6cb76c523d6014d"} Jul 09 18:04:27.374808 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.374794 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" event={"ID":"02d39fbc-ea0c-43d7-a221-f825998aa6d8","Type":"ContainerStarted","Data":"c7258d4099ef743d8ddff85f34460d839121abecfed0462fb4d9f85d84458354"} Jul 09 18:04:27.374861 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.374849 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" event={"ID":"02d39fbc-ea0c-43d7-a221-f825998aa6d8","Type":"ContainerStarted","Data":"8bb7f4a31c9f2ad9aeb4b6cb2f695aab929899b0c156c9c43cb1eb271617f5bd"} Jul 09 18:04:27.374914 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.374901 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" event={"ID":"02d39fbc-ea0c-43d7-a221-f825998aa6d8","Type":"ContainerStarted","Data":"5e1902e98b45fbab068ea82c0422b392d404d6089fad3a7bb5e2865137bbd298"} Jul 09 18:04:27.383101 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.383067 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zs6xt" podStartSLOduration=4.023301964 podStartE2EDuration="23.383055769s" totalImagesPullingTime="19.359753805s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:27.382968935 +0000 UTC m=+23.848491359" watchObservedRunningTime="2026-07-09 18:04:27.383055769 +0000 UTC m=+23.848578180" Jul 09 18:04:27.383419 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.383394 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-168.ec2.internal" podStartSLOduration=22.383387628 podStartE2EDuration="22.383387628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:04:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:09.34248283 +0000 UTC m=+5.808005246" watchObservedRunningTime="2026-07-09 18:04:27.383387628 +0000 UTC m=+23.848910038" Jul 09 18:04:27.417395 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.417309 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2qp88" podStartSLOduration=8.280459172 podStartE2EDuration="23.417288617s" totalImagesPullingTime="15.136829445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:27.398962309 +0000 UTC m=+23.864484720" watchObservedRunningTime="2026-07-09 18:04:27.417288617 +0000 UTC m=+23.882811022" Jul 09 18:04:27.417660 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.417629 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tkxzm" podStartSLOduration=4.007728118 podStartE2EDuration="23.417619186s" totalImagesPullingTime="19.409891068s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:27.417094883 +0000 UTC m=+23.882617297" watchObservedRunningTime="2026-07-09 18:04:27.417619186 +0000 UTC m=+23.883141596" Jul 09 18:04:27.487496 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.487440 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kwgh9" podStartSLOduration=4.119017518 podStartE2EDuration="23.487421058s" totalImagesPullingTime="19.36840354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:27.468004945 +0000 UTC m=+23.933527357" watchObservedRunningTime="2026-07-09 18:04:27.487421058 +0000 UTC m=+23.952943470" Jul 09 18:04:27.487669 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.487598 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-klp4q" podStartSLOduration=4.127673107 podStartE2EDuration="23.487589725s" totalImagesPullingTime="19.359916618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:27.487567444 +0000 UTC m=+23.953089843" watchObservedRunningTime="2026-07-09 18:04:27.487589725 +0000 UTC m=+23.953112137" Jul 09 18:04:27.687115 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.687078 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:27.787299 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.787263 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:27.798545 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:27.798526 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zs6xt_c4cada06-9af5-4483-a7a4-eca67319ff23/node-ca/0.log" Jul 09 18:04:28.072620 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.072576 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:28.136845 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.136714 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:28.141459 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.141438 2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 18:04:28.191642 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.191615 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:28.191750 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.191615 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:28.191750 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:28.191718 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:28.191831 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:28.191789 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:28.250879 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.250846 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:28.318396 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.318369 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:28.380448 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.380408 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" event={"ID":"02d39fbc-ea0c-43d7-a221-f825998aa6d8","Type":"ContainerStarted","Data":"69d67363c6d86689f47e752c6a23dec17320969426b56688c81413db61c09ed0"} Jul 09 18:04:28.382039 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.382010 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hspnc" event={"ID":"3112e5e6-80ab-4355-b4cc-669776e66483","Type":"ContainerStarted","Data":"81aeb2530e6ab2ed34d6ca168ace0d1ecbd7741ca71fa6bb362ae2b72632c3d1"} Jul 09 18:04:28.384087 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.384064 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" event={"ID":"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8","Type":"ContainerStarted","Data":"808425170d5f5f6cccf27839373bd353eef91eaab030354421a7bb4731aca696"} Jul 09 18:04:28.399610 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.399556 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hspnc" podStartSLOduration=5.037080648 podStartE2EDuration="24.399541256s" totalImagesPullingTime="19.362460608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:28.399514275 +0000 UTC m=+24.865036686" watchObservedRunningTime="2026-07-09 18:04:28.399541256 +0000 UTC m=+24.865063668" Jul 09 18:04:28.522083 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.522039 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:28.622804 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.622721 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:28.745947 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.745917 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:28.846393 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:28.846363 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:29.041059 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:29.040971 2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T18:04:28.141455822Z","UUID":"6e261e65-e68c-49c7-b4f7-a584c734abb5","Handler":null,"Name":"","Endpoint":""} Jul 09 18:04:29.042528 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:29.042482 2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 18:04:29.042528 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:29.042521 2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 18:04:29.191632 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:29.191606 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:29.191785 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:29.191762 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:29.297489 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:29.297411 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:29.388988 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:29.388948 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" event={"ID":"b7a7bd06-ebf8-43f4-acf5-7a62b07b73c8","Type":"ContainerStarted","Data":"fda9a6bc5085bee1dd5ceaf9fadaa108f88b1c58e2175d7986c8efaf32d77628"} Jul 09 18:04:29.409081 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:29.409028 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4h27j" podStartSLOduration=3.844217912 podStartE2EDuration="25.409012158s" totalImagesPullingTime="21.564794246s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:04:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:29.408623246 +0000 UTC m=+25.874145649" watchObservedRunningTime="2026-07-09 18:04:29.409012158 +0000 UTC m=+25.874534587" Jul 09 18:04:29.455037 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:29.455011 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:30.168885 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:30.168854 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2qp88" Jul 09 18:04:30.169477 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:30.169452 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2qp88" Jul 09 18:04:30.191813 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:30.191789 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:30.191945 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:30.191896 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:30.192017 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:30.191984 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:30.192101 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:30.192078 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:30.395761 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:30.395727 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" event={"ID":"02d39fbc-ea0c-43d7-a221-f825998aa6d8","Type":"ContainerStarted","Data":"b4daeede6f23a19d5f9075ffa3c740136652429ded01fbe21d12693bb1797e88"} Jul 09 18:04:30.431635 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:30.431557 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:31.050568 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:31.050541 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2qp88" Jul 09 18:04:31.051173 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:31.051152 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2qp88" Jul 09 18:04:31.190977 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:31.190946 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:31.191134 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:31.191085 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:31.354353 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:31.354324 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:31.811834 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:31.811801 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:32.191623 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:32.191443 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:32.191759 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:32.191475 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:32.191759 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:32.191738 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:32.191887 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:32.191764 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:32.402133 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:32.402097 2569 generic.go:356] "Generic (PLEG): container finished" podID="d084e88e-cb6f-4344-9d93-fee760d1d9c3" containerID="3afd97a8503f4d14f3f476a8b7e7e65e32ae526bf1f66d118ced2224c0b283b9" exitCode=0 Jul 09 18:04:32.402273 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:32.402159 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhp7" event={"ID":"d084e88e-cb6f-4344-9d93-fee760d1d9c3","Type":"ContainerDied","Data":"3afd97a8503f4d14f3f476a8b7e7e65e32ae526bf1f66d118ced2224c0b283b9"} Jul 09 18:04:32.406144 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:32.406119 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" event={"ID":"02d39fbc-ea0c-43d7-a221-f825998aa6d8","Type":"ContainerStarted","Data":"6bfc9794f97c199fb36e95ed274b0d8cebf56a9db422e661006dabee902da889"} Jul 09 18:04:32.406490 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:32.406474 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:32.421818 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:32.421800 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:32.439514 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:32.439459 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:32.456262 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:32.456221 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" podStartSLOduration=9.016730303 podStartE2EDuration="28.456208856s" totalImagesPullingTime="19.439478553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:32.455671476 +0000 UTC m=+28.921193887" watchObservedRunningTime="2026-07-09 18:04:32.456208856 +0000 UTC m=+28.921731267" Jul 09 18:04:32.528997 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:32.528963 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:32.630496 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:32.630463 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:32.736516 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:32.736439 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:32.866658 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:32.866625 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:33.131606 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:33.131567 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:33.191645 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:33.191612 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:33.191823 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:33.191776 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:33.409479 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:33.409405 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:33.409623 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:33.409495 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:33.425226 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:33.425198 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:04:34.192129 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:34.192053 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:34.192454 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:34.192159 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:34.192454 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:34.192190 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:34.192454 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:34.192261 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:34.326871 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:34.326835 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:34.413772 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:34.413734 2569 generic.go:356] "Generic (PLEG): container finished" podID="d084e88e-cb6f-4344-9d93-fee760d1d9c3" containerID="cdbac9fa08ddc5694b033ad2206bce50267c2ad278e883636a93223628be977c" exitCode=0 Jul 09 18:04:34.413917 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:34.413823 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhp7" event={"ID":"d084e88e-cb6f-4344-9d93-fee760d1d9c3","Type":"ContainerDied","Data":"cdbac9fa08ddc5694b033ad2206bce50267c2ad278e883636a93223628be977c"} Jul 09 18:04:34.491216 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:34.491136 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:34.675718 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:34.675660 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:34.893084 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:34.893045 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:35.074581 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:35.074554 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:35.191416 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:35.191351 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:35.191543 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:35.191463 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:36.081344 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:36.081312 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:36.191729 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:36.191699 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:36.191856 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:36.191814 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:36.191901 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:36.191869 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:36.191972 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:36.191954 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:36.213441 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:36.213418 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:36.317943 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:36.317904 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:36.419864 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:36.419796 2569 generic.go:356] "Generic (PLEG): container finished" podID="d084e88e-cb6f-4344-9d93-fee760d1d9c3" containerID="122cd67c58c2ffd2e1860c7b870f83e9104d232fa2a5dd6c06b227f4d74c54ed" exitCode=0 Jul 09 18:04:36.419979 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:36.419855 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhp7" event={"ID":"d084e88e-cb6f-4344-9d93-fee760d1d9c3","Type":"ContainerDied","Data":"122cd67c58c2ffd2e1860c7b870f83e9104d232fa2a5dd6c06b227f4d74c54ed"} Jul 09 18:04:36.497680 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:36.497635 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:36.679767 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:36.679638 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:36.901869 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:36.901809 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:37.032590 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:37.032556 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:37.135112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:37.135078 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:37.191270 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:37.191233 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:37.191420 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:37.191350 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:37.819371 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:37.819337 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs\") pod \"network-metrics-daemon-cr5qg\" (UID: \"25941c59-99b0-40e3-98fd-687a15e8da9e\") " pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:37.819614 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:37.819389 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:37.819614 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:37.819457 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:04:37.819614 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:37.819509 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 18:04:37.819614 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:37.819522 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs podName:25941c59-99b0-40e3-98fd-687a15e8da9e nodeName:}" failed. No retries permitted until 2026-07-09 18:05:09.819502327 +0000 UTC m=+66.285024716 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs") pod "network-metrics-daemon-cr5qg" (UID: "25941c59-99b0-40e3-98fd-687a15e8da9e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 18:04:37.819614 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:37.819565 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret podName:e1661ba5-6bd9-484b-97a0-8b28cb87e456 nodeName:}" failed. No retries permitted until 2026-07-09 18:05:09.819548534 +0000 UTC m=+66.285070929 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret") pod "global-pull-secret-syncer-sq92v" (UID: "e1661ba5-6bd9-484b-97a0-8b28cb87e456") : object "kube-system"/"original-pull-secret" not registered Jul 09 18:04:37.920463 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:37.920429 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qqrb\" (UniqueName: \"kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb\") pod \"network-check-target-lntgg\" (UID: \"aa5fad78-82ce-49bb-a50e-2b178586bdaf\") " pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:37.920633 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:37.920613 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 18:04:37.920682 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:37.920636 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 18:04:37.920682 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:37.920649 2569 projected.go:196] Error preparing data for projected volume kube-api-access-4qqrb for pod openshift-network-diagnostics/network-check-target-lntgg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:04:37.920799 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:37.920723 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb podName:aa5fad78-82ce-49bb-a50e-2b178586bdaf nodeName:}" failed. No retries permitted until 2026-07-09 18:05:09.920706753 +0000 UTC m=+66.386229157 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4qqrb" (UniqueName: "kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb") pod "network-check-target-lntgg" (UID: "aa5fad78-82ce-49bb-a50e-2b178586bdaf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 18:04:38.084838 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:38.084765 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:38.191496 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:38.191463 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:38.191496 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:38.191501 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:38.192051 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:38.191601 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:38.192051 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:38.192023 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:38.347280 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:38.347180 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:38.478997 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:38.478967 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:38.681344 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:38.681253 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:38.867462 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:38.867412 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:39.069392 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:39.069362 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:39.191807 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:39.191775 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:39.192233 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:39.191911 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:40.191582 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:40.191545 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:40.191771 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:40.191549 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:40.191771 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:40.191676 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:40.191771 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:40.191757 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:40.272290 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:40.272256 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:40.490803 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:40.490725 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:40.695021 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:40.694991 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:40.919349 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:40.919324 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:41.191121 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:41.191091 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:41.191301 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:41.191198 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:41.243625 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:41.243592 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:41.408671 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:41.408633 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:41.434295 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:41.434265 2569 generic.go:356] "Generic (PLEG): container finished" podID="d084e88e-cb6f-4344-9d93-fee760d1d9c3" containerID="6ac991c158b5fed89f3bb6e37d1c47c4348b4a9ebfd302ccbb1f2f2c8e85ac60" exitCode=0 Jul 09 18:04:41.434435 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:41.434328 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhp7" event={"ID":"d084e88e-cb6f-4344-9d93-fee760d1d9c3","Type":"ContainerDied","Data":"6ac991c158b5fed89f3bb6e37d1c47c4348b4a9ebfd302ccbb1f2f2c8e85ac60"} Jul 09 18:04:41.910799 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:41.910763 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:42.107916 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:42.107878 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:42.193879 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:42.193804 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:42.193879 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:42.193835 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:42.194050 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:42.193900 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:42.194050 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:42.194024 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:42.439673 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:42.439636 2569 generic.go:356] "Generic (PLEG): container finished" podID="d084e88e-cb6f-4344-9d93-fee760d1d9c3" containerID="cc161badc4e862427d8af13ffac752e6ae6dffc2ff764558649af29ae4209708" exitCode=0 Jul 09 18:04:42.440082 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:42.439716 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhp7" event={"ID":"d084e88e-cb6f-4344-9d93-fee760d1d9c3","Type":"ContainerDied","Data":"cc161badc4e862427d8af13ffac752e6ae6dffc2ff764558649af29ae4209708"} Jul 09 18:04:42.767126 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:42.767090 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:42.894805 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:42.894772 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:43.190095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:43.190008 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:43.191071 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:43.191055 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:43.191187 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:43.191167 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:43.445874 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:43.445787 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xvhp7" event={"ID":"d084e88e-cb6f-4344-9d93-fee760d1d9c3","Type":"ContainerStarted","Data":"feb0aa627918b566397be10fdfb31ba2ae20e011a0b068117c6d25307c4f4f3b"} Jul 09 18:04:44.194536 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:44.194504 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:44.194701 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:44.194504 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:44.194701 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:44.194598 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:44.194701 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:44.194664 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:44.295394 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:44.295351 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:45.191251 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:45.191205 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:45.191633 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:45.191328 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:46.191031 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:46.190993 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:46.191031 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:46.191014 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:46.191261 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:46.191108 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:46.191307 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:46.191254 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:47.191514 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:47.191477 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:47.191901 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:47.191594 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:48.191383 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:48.191348 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:48.191553 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:48.191462 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:48.191553 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:48.191515 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:48.191868 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:48.191612 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:49.191161 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:49.191122 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:49.191413 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:49.191242 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:50.191089 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:50.191044 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:50.191487 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:50.191160 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:50.191487 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:50.191214 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:50.191487 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:50.191308 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:51.191208 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:51.191173 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:51.191726 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:51.191294 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:52.191768 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:52.191732 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:52.192189 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:52.191779 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:52.192189 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:52.191847 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:52.192189 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:52.191980 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:53.191556 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:53.191527 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:53.191751 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:53.191627 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:53.738138 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:53.738104 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:53.897173 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:53.897136 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:54.193257 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:54.193211 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:54.193716 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:54.193665 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:54.193840 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:54.193547 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:54.193980 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:54.193965 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:54.262979 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:54.262925 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:54.418547 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:54.418507 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:54.661546 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:54.661512 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:55.049396 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:55.049364 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:55.148701 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:55.148669 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:55.191260 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:55.191222 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:55.191484 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:55.191343 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:55.445187 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:55.445123 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xvhp7" podStartSLOduration=28.333894005 podStartE2EDuration="51.445102805s" totalImagesPullingTime="23.1112088s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:04:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:04:43.489969501 +0000 UTC m=+39.955491911" watchObservedRunningTime="2026-07-09 18:04:55.445102805 +0000 UTC m=+51.910625217" Jul 09 18:04:55.446154 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:55.446127 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sq92v"] Jul 09 18:04:55.446303 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:55.446246 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:55.446426 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:55.446355 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:55.446938 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:55.446859 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cr5qg"] Jul 09 18:04:55.447645 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:55.447617 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lntgg"] Jul 09 18:04:55.447776 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:55.447734 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:55.447860 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:55.447834 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:55.472731 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:55.472706 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:55.472893 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:55.472830 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:55.849480 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:55.849319 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:55.946957 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:55.946892 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:57.191867 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:57.191826 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:57.192337 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:57.191928 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:57.192337 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:57.191932 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:57.192337 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:57.191952 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:57.192337 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:57.192025 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:57.192337 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:57.192087 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:59.026308 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.026272 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:59.127844 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.127809 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:59.191674 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.191634 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:04:59.191674 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.191633 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:04:59.191947 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.191634 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:04:59.191947 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:59.191763 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sq92v" podUID="e1661ba5-6bd9-484b-97a0-8b28cb87e456" Jul 09 18:04:59.191947 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:59.191843 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lntgg" podUID="aa5fad78-82ce-49bb-a50e-2b178586bdaf" Jul 09 18:04:59.191947 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:04:59.191918 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cr5qg" podUID="25941c59-99b0-40e3-98fd-687a15e8da9e" Jul 09 18:04:59.238163 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.238136 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:59.350864 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.350790 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-168.ec2.internal" event="NodeReady" Jul 09 18:04:59.350985 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.350928 2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 18:04:59.355150 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.355126 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:59.414363 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.414327 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5pc88"] Jul 09 18:04:59.416444 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.416424 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5pc88" Jul 09 18:04:59.418561 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.418536 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4jwbn"] Jul 09 18:04:59.419146 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.419124 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 18:04:59.419260 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.419167 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s5xfl\"" Jul 09 18:04:59.419260 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.419190 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 18:04:59.419260 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.419198 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 18:04:59.420615 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.420599 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.424321 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.424305 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 18:04:59.424659 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.424644 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 18:04:59.425259 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.424949 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 18:04:59.425259 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.424949 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4gbrj\"" Jul 09 18:04:59.425259 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.425019 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 18:04:59.434625 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.434606 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5pc88"] Jul 09 18:04:59.438924 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.438903 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4jwbn"] Jul 09 18:04:59.518825 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.518793 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mtt6h"] Jul 09 18:04:59.520655 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.520640 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mtt6h" Jul 09 18:04:59.523376 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.523355 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 18:04:59.523560 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.523547 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5876s\"" Jul 09 18:04:59.523777 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.523762 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 18:04:59.534915 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.534895 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mtt6h"] Jul 09 18:04:59.574021 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.573995 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d6cc2b0-cff5-4267-8cff-1d217e85385f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.574021 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.574021 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb68n\" (UniqueName: \"kubernetes.io/projected/f1984f61-d58e-4cc9-b883-3061273f2af5-kube-api-access-tb68n\") pod \"ingress-canary-5pc88\" (UID: \"f1984f61-d58e-4cc9-b883-3061273f2af5\") " pod="openshift-ingress-canary/ingress-canary-5pc88" Jul 09 18:04:59.574197 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.574050 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfh99\" (UniqueName: \"kubernetes.io/projected/6d6cc2b0-cff5-4267-8cff-1d217e85385f-kube-api-access-mfh99\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.574197 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.574077 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d6cc2b0-cff5-4267-8cff-1d217e85385f-data-volume\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.574197 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.574114 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6d6cc2b0-cff5-4267-8cff-1d217e85385f-crio-socket\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.574197 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.574136 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d6cc2b0-cff5-4267-8cff-1d217e85385f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.574197 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.574150 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1984f61-d58e-4cc9-b883-3061273f2af5-cert\") pod \"ingress-canary-5pc88\" (UID: \"f1984f61-d58e-4cc9-b883-3061273f2af5\") " pod="openshift-ingress-canary/ingress-canary-5pc88" Jul 09 18:04:59.675377 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.675289 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6d6cc2b0-cff5-4267-8cff-1d217e85385f-crio-socket\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.675377 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.675330 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d6cc2b0-cff5-4267-8cff-1d217e85385f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.675377 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.675357 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1984f61-d58e-4cc9-b883-3061273f2af5-cert\") pod \"ingress-canary-5pc88\" (UID: \"f1984f61-d58e-4cc9-b883-3061273f2af5\") " pod="openshift-ingress-canary/ingress-canary-5pc88" Jul 09 18:04:59.675755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.675415 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6d6cc2b0-cff5-4267-8cff-1d217e85385f-crio-socket\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.675755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.675422 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d6cc2b0-cff5-4267-8cff-1d217e85385f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.675755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.675485 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb68n\" (UniqueName: \"kubernetes.io/projected/f1984f61-d58e-4cc9-b883-3061273f2af5-kube-api-access-tb68n\") pod \"ingress-canary-5pc88\" (UID: \"f1984f61-d58e-4cc9-b883-3061273f2af5\") " pod="openshift-ingress-canary/ingress-canary-5pc88" Jul 09 18:04:59.675755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.675509 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4888cd81-969f-4aa9-ab7a-d9ca367fc111-tmp-dir\") pod \"dns-default-mtt6h\" (UID: \"4888cd81-969f-4aa9-ab7a-d9ca367fc111\") " pod="openshift-dns/dns-default-mtt6h" Jul 09 18:04:59.675755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.675554 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfh99\" (UniqueName: \"kubernetes.io/projected/6d6cc2b0-cff5-4267-8cff-1d217e85385f-kube-api-access-mfh99\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.675755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.675580 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj4gz\" (UniqueName: \"kubernetes.io/projected/4888cd81-969f-4aa9-ab7a-d9ca367fc111-kube-api-access-dj4gz\") pod \"dns-default-mtt6h\" (UID: \"4888cd81-969f-4aa9-ab7a-d9ca367fc111\") " pod="openshift-dns/dns-default-mtt6h" Jul 09 18:04:59.675755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.675739 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d6cc2b0-cff5-4267-8cff-1d217e85385f-data-volume\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.676092 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.675787 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4888cd81-969f-4aa9-ab7a-d9ca367fc111-config-volume\") pod \"dns-default-mtt6h\" (UID: \"4888cd81-969f-4aa9-ab7a-d9ca367fc111\") " pod="openshift-dns/dns-default-mtt6h" Jul 09 18:04:59.676092 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.675813 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4888cd81-969f-4aa9-ab7a-d9ca367fc111-metrics-tls\") pod \"dns-default-mtt6h\" (UID: \"4888cd81-969f-4aa9-ab7a-d9ca367fc111\") " pod="openshift-dns/dns-default-mtt6h" Jul 09 18:04:59.676092 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.675997 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d6cc2b0-cff5-4267-8cff-1d217e85385f-data-volume\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.676092 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.676087 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d6cc2b0-cff5-4267-8cff-1d217e85385f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.679472 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.679452 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d6cc2b0-cff5-4267-8cff-1d217e85385f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.679595 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.679499 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1984f61-d58e-4cc9-b883-3061273f2af5-cert\") pod \"ingress-canary-5pc88\" (UID: \"f1984f61-d58e-4cc9-b883-3061273f2af5\") " pod="openshift-ingress-canary/ingress-canary-5pc88" Jul 09 18:04:59.685341 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.685315 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb68n\" (UniqueName: \"kubernetes.io/projected/f1984f61-d58e-4cc9-b883-3061273f2af5-kube-api-access-tb68n\") pod \"ingress-canary-5pc88\" (UID: \"f1984f61-d58e-4cc9-b883-3061273f2af5\") " pod="openshift-ingress-canary/ingress-canary-5pc88" Jul 09 18:04:59.686382 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.686362 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfh99\" (UniqueName: \"kubernetes.io/projected/6d6cc2b0-cff5-4267-8cff-1d217e85385f-kube-api-access-mfh99\") pod \"insights-runtime-extractor-4jwbn\" (UID: \"6d6cc2b0-cff5-4267-8cff-1d217e85385f\") " pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.728342 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.728288 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5pc88" Jul 09 18:04:59.730604 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.730549 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:59.732012 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.731993 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4jwbn" Jul 09 18:04:59.777967 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.777933 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4888cd81-969f-4aa9-ab7a-d9ca367fc111-tmp-dir\") pod \"dns-default-mtt6h\" (UID: \"4888cd81-969f-4aa9-ab7a-d9ca367fc111\") " pod="openshift-dns/dns-default-mtt6h" Jul 09 18:04:59.778149 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.778000 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj4gz\" (UniqueName: \"kubernetes.io/projected/4888cd81-969f-4aa9-ab7a-d9ca367fc111-kube-api-access-dj4gz\") pod \"dns-default-mtt6h\" (UID: \"4888cd81-969f-4aa9-ab7a-d9ca367fc111\") " pod="openshift-dns/dns-default-mtt6h" Jul 09 18:04:59.778149 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.778039 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4888cd81-969f-4aa9-ab7a-d9ca367fc111-config-volume\") pod \"dns-default-mtt6h\" (UID: \"4888cd81-969f-4aa9-ab7a-d9ca367fc111\") " pod="openshift-dns/dns-default-mtt6h" Jul 09 18:04:59.778149 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.778064 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4888cd81-969f-4aa9-ab7a-d9ca367fc111-metrics-tls\") pod \"dns-default-mtt6h\" (UID: \"4888cd81-969f-4aa9-ab7a-d9ca367fc111\") " pod="openshift-dns/dns-default-mtt6h" Jul 09 18:04:59.778748 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.778717 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4888cd81-969f-4aa9-ab7a-d9ca367fc111-tmp-dir\") pod \"dns-default-mtt6h\" (UID: \"4888cd81-969f-4aa9-ab7a-d9ca367fc111\") " pod="openshift-dns/dns-default-mtt6h" Jul 09 18:04:59.778971 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.778946 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4888cd81-969f-4aa9-ab7a-d9ca367fc111-config-volume\") pod \"dns-default-mtt6h\" (UID: \"4888cd81-969f-4aa9-ab7a-d9ca367fc111\") " pod="openshift-dns/dns-default-mtt6h" Jul 09 18:04:59.781105 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.781081 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4888cd81-969f-4aa9-ab7a-d9ca367fc111-metrics-tls\") pod \"dns-default-mtt6h\" (UID: \"4888cd81-969f-4aa9-ab7a-d9ca367fc111\") " pod="openshift-dns/dns-default-mtt6h" Jul 09 18:04:59.788267 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.788235 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj4gz\" (UniqueName: \"kubernetes.io/projected/4888cd81-969f-4aa9-ab7a-d9ca367fc111-kube-api-access-dj4gz\") pod \"dns-default-mtt6h\" (UID: \"4888cd81-969f-4aa9-ab7a-d9ca367fc111\") " pod="openshift-dns/dns-default-mtt6h" Jul 09 18:04:59.823196 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.823138 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:04:59.829266 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.829022 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mtt6h" Jul 09 18:04:59.908119 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.905707 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5pc88"] Jul 09 18:04:59.908119 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.905815 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4jwbn"] Jul 09 18:04:59.997080 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:04:59.997058 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mtt6h"] Jul 09 18:04:59.998030 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:04:59.997999 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4888cd81_969f_4aa9_ab7a_d9ca367fc111.slice/crio-6feb5f860373a2ef9a21f999637be2885b48e48e6ef9f03ab383c560ab4ecefb WatchSource:0}: Error finding container 6feb5f860373a2ef9a21f999637be2885b48e48e6ef9f03ab383c560ab4ecefb: Status 404 returned error can't find the container with id 6feb5f860373a2ef9a21f999637be2885b48e48e6ef9f03ab383c560ab4ecefb Jul 09 18:05:00.488540 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:00.488499 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mtt6h" event={"ID":"4888cd81-969f-4aa9-ab7a-d9ca367fc111","Type":"ContainerStarted","Data":"6feb5f860373a2ef9a21f999637be2885b48e48e6ef9f03ab383c560ab4ecefb"} Jul 09 18:05:00.493385 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:00.493347 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4jwbn" event={"ID":"6d6cc2b0-cff5-4267-8cff-1d217e85385f","Type":"ContainerStarted","Data":"5d610e8bec6aa506c8055b9a4ecf34c94dbedd20407b6f4fdeb420b869524aeb"} Jul 09 18:05:00.494935 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:00.494903 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5pc88" event={"ID":"f1984f61-d58e-4cc9-b883-3061273f2af5","Type":"ContainerStarted","Data":"dfdca2b0473e5b7d81a26ad4fa862263aaf39df74f358b01ac98a264de55d519"} Jul 09 18:05:01.191725 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:01.191667 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:05:01.192120 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:01.192104 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:05:01.192515 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:01.192444 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:05:01.194733 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:01.194713 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 18:05:01.194900 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:01.194881 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 18:05:01.195819 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:01.195800 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jfsz2\"" Jul 09 18:05:01.196072 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:01.196034 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 18:05:01.196365 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:01.196348 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sjz5q\"" Jul 09 18:05:01.196575 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:01.196555 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 18:05:01.242952 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:01.242915 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:01.319762 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:01.319678 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:02.695906 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:02.695875 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:02.875595 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:02.875563 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:05.382928 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.382896 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:05.433301 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.433274 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sjxbv" Jul 09 18:05:05.528723 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.528673 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-pkjvw"] Jul 09 18:05:05.538582 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.538565 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" Jul 09 18:05:05.577341 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.577322 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 09 18:05:05.578012 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.577984 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xrvtf\"" Jul 09 18:05:05.578164 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.578145 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 18:05:05.578217 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.578162 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 09 18:05:05.578217 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.578147 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 18:05:05.579825 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.579757 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 18:05:05.588220 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.588198 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-pkjvw"] Jul 09 18:05:05.626534 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.626498 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/53f1ad17-1807-49b3-87b1-148ea3b3c791-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-pkjvw\" (UID: \"53f1ad17-1807-49b3-87b1-148ea3b3c791\") " pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" Jul 09 18:05:05.626534 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.626537 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfwcm\" (UniqueName: \"kubernetes.io/projected/53f1ad17-1807-49b3-87b1-148ea3b3c791-kube-api-access-mfwcm\") pod \"prometheus-operator-64785b6675-pkjvw\" (UID: \"53f1ad17-1807-49b3-87b1-148ea3b3c791\") " pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" Jul 09 18:05:05.626789 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.626564 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53f1ad17-1807-49b3-87b1-148ea3b3c791-metrics-client-ca\") pod \"prometheus-operator-64785b6675-pkjvw\" (UID: \"53f1ad17-1807-49b3-87b1-148ea3b3c791\") " pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" Jul 09 18:05:05.626789 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.626643 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/53f1ad17-1807-49b3-87b1-148ea3b3c791-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-pkjvw\" (UID: \"53f1ad17-1807-49b3-87b1-148ea3b3c791\") " pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" Jul 09 18:05:05.694870 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.694837 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:05.727083 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.727047 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/53f1ad17-1807-49b3-87b1-148ea3b3c791-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-pkjvw\" (UID: \"53f1ad17-1807-49b3-87b1-148ea3b3c791\") " pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" Jul 09 18:05:05.727190 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.727149 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/53f1ad17-1807-49b3-87b1-148ea3b3c791-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-pkjvw\" (UID: \"53f1ad17-1807-49b3-87b1-148ea3b3c791\") " pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" Jul 09 18:05:05.727190 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.727182 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfwcm\" (UniqueName: \"kubernetes.io/projected/53f1ad17-1807-49b3-87b1-148ea3b3c791-kube-api-access-mfwcm\") pod \"prometheus-operator-64785b6675-pkjvw\" (UID: \"53f1ad17-1807-49b3-87b1-148ea3b3c791\") " pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" Jul 09 18:05:05.727272 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.727217 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53f1ad17-1807-49b3-87b1-148ea3b3c791-metrics-client-ca\") pod \"prometheus-operator-64785b6675-pkjvw\" (UID: \"53f1ad17-1807-49b3-87b1-148ea3b3c791\") " pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" Jul 09 18:05:05.728095 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.728045 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53f1ad17-1807-49b3-87b1-148ea3b3c791-metrics-client-ca\") pod \"prometheus-operator-64785b6675-pkjvw\" (UID: \"53f1ad17-1807-49b3-87b1-148ea3b3c791\") " pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" Jul 09 18:05:05.730768 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.730744 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/53f1ad17-1807-49b3-87b1-148ea3b3c791-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-pkjvw\" (UID: \"53f1ad17-1807-49b3-87b1-148ea3b3c791\") " pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" Jul 09 18:05:05.730882 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.730827 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/53f1ad17-1807-49b3-87b1-148ea3b3c791-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-pkjvw\" (UID: \"53f1ad17-1807-49b3-87b1-148ea3b3c791\") " pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" Jul 09 18:05:05.741738 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.741708 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfwcm\" (UniqueName: \"kubernetes.io/projected/53f1ad17-1807-49b3-87b1-148ea3b3c791-kube-api-access-mfwcm\") pod \"prometheus-operator-64785b6675-pkjvw\" (UID: \"53f1ad17-1807-49b3-87b1-148ea3b3c791\") " pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" Jul 09 18:05:05.804792 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.804754 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:05.860126 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.860089 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" Jul 09 18:05:05.932465 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:05.932437 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:06.007108 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:06.007078 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-pkjvw"] Jul 09 18:05:06.010381 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:05:06.010352 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53f1ad17_1807_49b3_87b1_148ea3b3c791.slice/crio-b30065d4b0e07d0717830a4eee354c4b63d209f543b9c0395f720828997399a9 WatchSource:0}: Error finding container b30065d4b0e07d0717830a4eee354c4b63d209f543b9c0395f720828997399a9: Status 404 returned error can't find the container with id b30065d4b0e07d0717830a4eee354c4b63d209f543b9c0395f720828997399a9 Jul 09 18:05:06.513487 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:06.513455 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5pc88" event={"ID":"f1984f61-d58e-4cc9-b883-3061273f2af5","Type":"ContainerStarted","Data":"7a74b6d80c1c0c408b87af11cd8b81cc31887c6d60bad8644a229fd72a291470"} Jul 09 18:05:06.515337 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:06.515304 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mtt6h" event={"ID":"4888cd81-969f-4aa9-ab7a-d9ca367fc111","Type":"ContainerStarted","Data":"f3cae2f08c6005358ce7441a878dc19297b1ec4657379bcc78744f7e7b83a16b"} Jul 09 18:05:06.515460 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:06.515366 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mtt6h" event={"ID":"4888cd81-969f-4aa9-ab7a-d9ca367fc111","Type":"ContainerStarted","Data":"b015e54ebaf37b3c68518f11091decf84b1e031c4916546c09a3a5191a9606a7"} Jul 09 18:05:06.515460 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:06.515391 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mtt6h" Jul 09 18:05:06.516806 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:06.516780 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4jwbn" event={"ID":"6d6cc2b0-cff5-4267-8cff-1d217e85385f","Type":"ContainerStarted","Data":"4c7556198ccce69b6c05f3e4b62922bf0c5e5a131c67f032055cba3dca3c34cf"} Jul 09 18:05:06.517880 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:06.517856 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" event={"ID":"53f1ad17-1807-49b3-87b1-148ea3b3c791","Type":"ContainerStarted","Data":"b30065d4b0e07d0717830a4eee354c4b63d209f543b9c0395f720828997399a9"} Jul 09 18:05:06.581303 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:06.581248 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5pc88" podStartSLOduration=2.002478257 podStartE2EDuration="7.581230454s" totalImagesPullingTime="5.578752197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:06.544792999 +0000 UTC m=+63.010315411" watchObservedRunningTime="2026-07-09 18:05:06.581230454 +0000 UTC m=+63.046752870" Jul 09 18:05:06.581406 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:06.581362 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mtt6h" podStartSLOduration=2.096396501 podStartE2EDuration="7.581354825s" totalImagesPullingTime="5.484958324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:06.580720388 +0000 UTC m=+63.046242796" watchObservedRunningTime="2026-07-09 18:05:06.581354825 +0000 UTC m=+63.046877237" Jul 09 18:05:07.524374 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:07.524230 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4jwbn" event={"ID":"6d6cc2b0-cff5-4267-8cff-1d217e85385f","Type":"ContainerStarted","Data":"274e98e20e55c51a9c159dca17184a84109d8cec5599645a525a54005de972d5"} Jul 09 18:05:08.049208 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:08.049113 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:08.530607 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:08.530564 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" event={"ID":"53f1ad17-1807-49b3-87b1-148ea3b3c791","Type":"ContainerStarted","Data":"ddc53c399c54fca76698ed390c3ab64510e35d07e80f76a94d0df234c8bf9c26"} Jul 09 18:05:08.531045 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:08.530630 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" event={"ID":"53f1ad17-1807-49b3-87b1-148ea3b3c791","Type":"ContainerStarted","Data":"e4019a5cc87a7c988155da4559f03d1ca158ef69ba7be46b93d6b3a4525e68b3"} Jul 09 18:05:08.560520 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:08.560461 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-pkjvw" podStartSLOduration=1.8919940830000002 podStartE2EDuration="3.560443514s" totalImagesPullingTime="1.668449431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:05:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:08.559837939 +0000 UTC m=+65.025360372" watchObservedRunningTime="2026-07-09 18:05:08.560443514 +0000 UTC m=+65.025965925" Jul 09 18:05:09.536259 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.536221 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4jwbn" event={"ID":"6d6cc2b0-cff5-4267-8cff-1d217e85385f","Type":"ContainerStarted","Data":"b6f9c4aefcbca37f834e7d04f17cb18f8b3983e7bf3db23551ac24d81c94e7e9"} Jul 09 18:05:09.603811 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.603757 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4jwbn" podStartSLOduration=1.29039031 podStartE2EDuration="10.603738615s" totalImagesPullingTime="9.313348305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:04:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:09.603007464 +0000 UTC m=+66.068529876" watchObservedRunningTime="2026-07-09 18:05:09.603738615 +0000 UTC m=+66.069261026" Jul 09 18:05:09.860982 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.860941 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs\") pod \"network-metrics-daemon-cr5qg\" (UID: \"25941c59-99b0-40e3-98fd-687a15e8da9e\") " pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:05:09.860982 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.860986 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:05:09.864516 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.864492 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 18:05:09.864628 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.864599 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 18:05:09.874471 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.874447 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25941c59-99b0-40e3-98fd-687a15e8da9e-metrics-certs\") pod \"network-metrics-daemon-cr5qg\" (UID: \"25941c59-99b0-40e3-98fd-687a15e8da9e\") " pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:05:09.874583 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.874528 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1661ba5-6bd9-484b-97a0-8b28cb87e456-original-pull-secret\") pod \"global-pull-secret-syncer-sq92v\" (UID: \"e1661ba5-6bd9-484b-97a0-8b28cb87e456\") " pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:05:09.909767 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.909739 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sq92v" Jul 09 18:05:09.929339 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.929318 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sjz5q\"" Jul 09 18:05:09.935057 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.935033 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cr5qg" Jul 09 18:05:09.962267 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.962185 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qqrb\" (UniqueName: \"kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb\") pod \"network-check-target-lntgg\" (UID: \"aa5fad78-82ce-49bb-a50e-2b178586bdaf\") " pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:05:09.966835 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.966814 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 18:05:09.976853 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.976596 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 18:05:09.986385 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:09.986358 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qqrb\" (UniqueName: \"kubernetes.io/projected/aa5fad78-82ce-49bb-a50e-2b178586bdaf-kube-api-access-4qqrb\") pod \"network-check-target-lntgg\" (UID: \"aa5fad78-82ce-49bb-a50e-2b178586bdaf\") " pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:05:10.052008 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:10.051979 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sq92v"] Jul 09 18:05:10.057610 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:05:10.057585 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1661ba5_6bd9_484b_97a0_8b28cb87e456.slice/crio-018ac02ab0e8e4675c54b242a25f22a5b4e85676b0b6253468ba9adc826d1b79 WatchSource:0}: Error finding container 018ac02ab0e8e4675c54b242a25f22a5b4e85676b0b6253468ba9adc826d1b79: Status 404 returned error can't find the container with id 018ac02ab0e8e4675c54b242a25f22a5b4e85676b0b6253468ba9adc826d1b79 Jul 09 18:05:10.080300 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:10.080251 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cr5qg"] Jul 09 18:05:10.084367 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:05:10.084342 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25941c59_99b0_40e3_98fd_687a15e8da9e.slice/crio-40b582e525474fe7da8433e030bf19e9b48fb94fab735165cd4393022b38bd03 WatchSource:0}: Error finding container 40b582e525474fe7da8433e030bf19e9b48fb94fab735165cd4393022b38bd03: Status 404 returned error can't find the container with id 40b582e525474fe7da8433e030bf19e9b48fb94fab735165cd4393022b38bd03 Jul 09 18:05:10.111297 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:10.111197 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:10.211307 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:10.211264 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:10.221089 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:10.221061 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jfsz2\"" Jul 09 18:05:10.228233 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:10.228212 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:05:10.361352 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:10.361298 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lntgg"] Jul 09 18:05:10.363043 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:05:10.363019 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa5fad78_82ce_49bb_a50e_2b178586bdaf.slice/crio-b08d934ddc3469edb61afcc3964cd79bf87338257a3e622ab599c5ff2c53a728 WatchSource:0}: Error finding container b08d934ddc3469edb61afcc3964cd79bf87338257a3e622ab599c5ff2c53a728: Status 404 returned error can't find the container with id b08d934ddc3469edb61afcc3964cd79bf87338257a3e622ab599c5ff2c53a728 Jul 09 18:05:10.541338 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:10.541302 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lntgg" event={"ID":"aa5fad78-82ce-49bb-a50e-2b178586bdaf","Type":"ContainerStarted","Data":"b08d934ddc3469edb61afcc3964cd79bf87338257a3e622ab599c5ff2c53a728"} Jul 09 18:05:10.542408 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:10.542377 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cr5qg" event={"ID":"25941c59-99b0-40e3-98fd-687a15e8da9e","Type":"ContainerStarted","Data":"40b582e525474fe7da8433e030bf19e9b48fb94fab735165cd4393022b38bd03"} Jul 09 18:05:10.543428 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:10.543407 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sq92v" event={"ID":"e1661ba5-6bd9-484b-97a0-8b28cb87e456","Type":"ContainerStarted","Data":"018ac02ab0e8e4675c54b242a25f22a5b4e85676b0b6253468ba9adc826d1b79"} Jul 09 18:05:10.722976 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:10.722879 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:10.844985 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:10.844949 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:11.083485 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.083423 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:11.164495 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.164465 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8"] Jul 09 18:05:11.194703 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.194651 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8"] Jul 09 18:05:11.194876 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.194831 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:11.199149 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.199125 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 09 18:05:11.199314 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.199234 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5lvhl\"" Jul 09 18:05:11.199387 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.199367 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 09 18:05:11.210872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.210848 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-7lqhs"] Jul 09 18:05:11.233052 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.233014 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.233614 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.233592 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-7lqhs"] Jul 09 18:05:11.235926 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.235905 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 18:05:11.236139 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.236123 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 18:05:11.236428 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.236410 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-cz2wz\"" Jul 09 18:05:11.236894 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.236875 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 18:05:11.244510 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.244483 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dszrz"] Jul 09 18:05:11.245633 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.245605 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:11.267613 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.267577 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.270264 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.270243 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 18:05:11.270409 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.270305 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 18:05:11.270581 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.270566 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jnvlp\"" Jul 09 18:05:11.270638 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.270576 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 18:05:11.374819 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.374681 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05a00921-ffc8-419d-b9e1-e76d8c6580ee-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-brlv8\" (UID: \"05a00921-ffc8-419d-b9e1-e76d8c6580ee\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:11.374819 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.374787 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-textfile\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.374819 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.374819 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05a00921-ffc8-419d-b9e1-e76d8c6580ee-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-brlv8\" (UID: \"05a00921-ffc8-419d-b9e1-e76d8c6580ee\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:11.375082 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.374845 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/05a00921-ffc8-419d-b9e1-e76d8c6580ee-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-brlv8\" (UID: \"05a00921-ffc8-419d-b9e1-e76d8c6580ee\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:11.375082 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.374910 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37199f74-3a3d-4d96-9b8d-7c248d641d11-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.375082 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.374950 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8jcq\" (UniqueName: \"kubernetes.io/projected/37199f74-3a3d-4d96-9b8d-7c248d641d11-kube-api-access-c8jcq\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.375082 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.374976 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbdd2842-6040-47e2-964f-613c8026aebf-metrics-client-ca\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.375082 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.374998 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37199f74-3a3d-4d96-9b8d-7c248d641d11-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.375082 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.375014 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.375082 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.375059 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-tls\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.375427 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.375110 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/37199f74-3a3d-4d96-9b8d-7c248d641d11-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.375427 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.375139 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bbdd2842-6040-47e2-964f-613c8026aebf-root\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.375427 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.375175 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbdd2842-6040-47e2-964f-613c8026aebf-sys\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.375427 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.375198 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-accelerators-collector-config\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.375427 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.375248 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpjs8\" (UniqueName: \"kubernetes.io/projected/bbdd2842-6040-47e2-964f-613c8026aebf-kube-api-access-rpjs8\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.375427 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.375266 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtg62\" (UniqueName: \"kubernetes.io/projected/05a00921-ffc8-419d-b9e1-e76d8c6580ee-kube-api-access-xtg62\") pod \"openshift-state-metrics-84bd68b896-brlv8\" (UID: \"05a00921-ffc8-419d-b9e1-e76d8c6580ee\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:11.375427 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.375326 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-wtmp\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.375427 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.375391 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37199f74-3a3d-4d96-9b8d-7c248d641d11-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.375427 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.375424 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/37199f74-3a3d-4d96-9b8d-7c248d641d11-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.393216 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.393180 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:11.476187 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.476130 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37199f74-3a3d-4d96-9b8d-7c248d641d11-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.476187 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.476194 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8jcq\" (UniqueName: \"kubernetes.io/projected/37199f74-3a3d-4d96-9b8d-7c248d641d11-kube-api-access-c8jcq\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.476423 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.476220 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbdd2842-6040-47e2-964f-613c8026aebf-metrics-client-ca\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.476576 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37199f74-3a3d-4d96-9b8d-7c248d641d11-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.476639 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.476667 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-tls\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.476753 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/37199f74-3a3d-4d96-9b8d-7c248d641d11-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.476783 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bbdd2842-6040-47e2-964f-613c8026aebf-root\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.476829 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbdd2842-6040-47e2-964f-613c8026aebf-sys\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.476863 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-accelerators-collector-config\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.476890 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpjs8\" (UniqueName: \"kubernetes.io/projected/bbdd2842-6040-47e2-964f-613c8026aebf-kube-api-access-rpjs8\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.476913 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtg62\" (UniqueName: \"kubernetes.io/projected/05a00921-ffc8-419d-b9e1-e76d8c6580ee-kube-api-access-xtg62\") pod \"openshift-state-metrics-84bd68b896-brlv8\" (UID: \"05a00921-ffc8-419d-b9e1-e76d8c6580ee\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.476948 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-wtmp\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.476981 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37199f74-3a3d-4d96-9b8d-7c248d641d11-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.477006 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/37199f74-3a3d-4d96-9b8d-7c248d641d11-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.477046 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05a00921-ffc8-419d-b9e1-e76d8c6580ee-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-brlv8\" (UID: \"05a00921-ffc8-419d-b9e1-e76d8c6580ee\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.477074 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-textfile\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.477118 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05a00921-ffc8-419d-b9e1-e76d8c6580ee-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-brlv8\" (UID: \"05a00921-ffc8-419d-b9e1-e76d8c6580ee\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:11.477872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.477251 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37199f74-3a3d-4d96-9b8d-7c248d641d11-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.478919 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.477302 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/05a00921-ffc8-419d-b9e1-e76d8c6580ee-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-brlv8\" (UID: \"05a00921-ffc8-419d-b9e1-e76d8c6580ee\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:11.478919 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.477332 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bbdd2842-6040-47e2-964f-613c8026aebf-root\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.478919 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.477398 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbdd2842-6040-47e2-964f-613c8026aebf-sys\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.478919 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:05:11.477438 2569 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 09 18:05:11.478919 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:05:11.477500 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05a00921-ffc8-419d-b9e1-e76d8c6580ee-openshift-state-metrics-tls podName:05a00921-ffc8-419d-b9e1-e76d8c6580ee nodeName:}" failed. No retries permitted until 2026-07-09 18:05:11.977480054 +0000 UTC m=+68.443002447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/05a00921-ffc8-419d-b9e1-e76d8c6580ee-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-brlv8" (UID: "05a00921-ffc8-419d-b9e1-e76d8c6580ee") : secret "openshift-state-metrics-tls" not found Jul 09 18:05:11.478919 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.477560 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/37199f74-3a3d-4d96-9b8d-7c248d641d11-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.478919 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.477908 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-accelerators-collector-config\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.478919 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.477569 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-textfile\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.478919 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.477140 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbdd2842-6040-47e2-964f-613c8026aebf-metrics-client-ca\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.478919 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.478076 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-wtmp\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.478919 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.478337 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05a00921-ffc8-419d-b9e1-e76d8c6580ee-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-brlv8\" (UID: \"05a00921-ffc8-419d-b9e1-e76d8c6580ee\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:11.478919 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.478453 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/37199f74-3a3d-4d96-9b8d-7c248d641d11-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.479590 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.479437 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37199f74-3a3d-4d96-9b8d-7c248d641d11-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.480807 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.480337 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-tls\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.481151 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.481122 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37199f74-3a3d-4d96-9b8d-7c248d641d11-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.481447 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.481417 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05a00921-ffc8-419d-b9e1-e76d8c6580ee-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-brlv8\" (UID: \"05a00921-ffc8-419d-b9e1-e76d8c6580ee\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:11.487355 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.487324 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbdd2842-6040-47e2-964f-613c8026aebf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.487632 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.487605 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpjs8\" (UniqueName: \"kubernetes.io/projected/bbdd2842-6040-47e2-964f-613c8026aebf-kube-api-access-rpjs8\") pod \"node-exporter-dszrz\" (UID: \"bbdd2842-6040-47e2-964f-613c8026aebf\") " pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.488177 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.488131 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8jcq\" (UniqueName: \"kubernetes.io/projected/37199f74-3a3d-4d96-9b8d-7c248d641d11-kube-api-access-c8jcq\") pod \"kube-state-metrics-785778ff74-7lqhs\" (UID: \"37199f74-3a3d-4d96-9b8d-7c248d641d11\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.489240 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.489216 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtg62\" (UniqueName: \"kubernetes.io/projected/05a00921-ffc8-419d-b9e1-e76d8c6580ee-kube-api-access-xtg62\") pod \"openshift-state-metrics-84bd68b896-brlv8\" (UID: \"05a00921-ffc8-419d-b9e1-e76d8c6580ee\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:11.542421 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.542383 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:11.544933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.544909 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" Jul 09 18:05:11.578596 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.578401 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dszrz" Jul 09 18:05:11.607807 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:05:11.607741 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbdd2842_6040_47e2_964f_613c8026aebf.slice/crio-d8e558cf69aa2d88cce6a34ede51195103b944fa3b41a65e60ca565241ebf998 WatchSource:0}: Error finding container d8e558cf69aa2d88cce6a34ede51195103b944fa3b41a65e60ca565241ebf998: Status 404 returned error can't find the container with id d8e558cf69aa2d88cce6a34ede51195103b944fa3b41a65e60ca565241ebf998 Jul 09 18:05:11.665565 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.665440 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:11.783065 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.783032 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-7lqhs"] Jul 09 18:05:11.786251 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:05:11.786215 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37199f74_3a3d_4d96_9b8d_7c248d641d11.slice/crio-bcab5de0617b265f2ed6a8775c10245930651a5e84e9c6eaf1b65df9ba7c1044 WatchSource:0}: Error finding container bcab5de0617b265f2ed6a8775c10245930651a5e84e9c6eaf1b65df9ba7c1044: Status 404 returned error can't find the container with id bcab5de0617b265f2ed6a8775c10245930651a5e84e9c6eaf1b65df9ba7c1044 Jul 09 18:05:11.807538 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.807500 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:11.982486 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.982449 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/05a00921-ffc8-419d-b9e1-e76d8c6580ee-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-brlv8\" (UID: \"05a00921-ffc8-419d-b9e1-e76d8c6580ee\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:11.985665 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:11.985611 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/05a00921-ffc8-419d-b9e1-e76d8c6580ee-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-brlv8\" (UID: \"05a00921-ffc8-419d-b9e1-e76d8c6580ee\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:12.105808 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.105775 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" Jul 09 18:05:12.116316 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.116245 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:12.218251 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.218175 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:12.239964 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.239929 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:05:12.278105 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.277743 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:05:12.278105 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.277919 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.286383 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286341 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.286532 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286407 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.286532 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286439 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.286532 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286468 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7678ee9-03ce-4b69-81f1-8101576a1350-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.286532 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286499 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7678ee9-03ce-4b69-81f1-8101576a1350-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.286532 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286527 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7678ee9-03ce-4b69-81f1-8101576a1350-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.286778 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286564 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6js9\" (UniqueName: \"kubernetes.io/projected/e7678ee9-03ce-4b69-81f1-8101576a1350-kube-api-access-z6js9\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.286778 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286597 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.286778 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286622 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 18:05:12.286778 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286747 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 18:05:12.286984 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286624 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-web-config\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.286984 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286907 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.286984 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286940 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7678ee9-03ce-4b69-81f1-8101576a1350-config-out\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.286984 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286946 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 18:05:12.286984 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.286971 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e7678ee9-03ce-4b69-81f1-8101576a1350-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.287232 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.287014 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-config-volume\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.287881 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.287276 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 18:05:12.287881 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.287378 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 18:05:12.287881 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.287499 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 18:05:12.288270 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.288113 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 18:05:12.289078 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.288866 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rqd7s\"" Jul 09 18:05:12.289078 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.288893 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 18:05:12.289078 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.289064 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 18:05:12.302936 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.300127 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8"] Jul 09 18:05:12.306323 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:05:12.306118 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05a00921_ffc8_419d_b9e1_e76d8c6580ee.slice/crio-f9c905bfbb00998dd5e4d7e05097ba8e66e42e806f905f9bcd7309a4b1aa7c51 WatchSource:0}: Error finding container f9c905bfbb00998dd5e4d7e05097ba8e66e42e806f905f9bcd7309a4b1aa7c51: Status 404 returned error can't find the container with id f9c905bfbb00998dd5e4d7e05097ba8e66e42e806f905f9bcd7309a4b1aa7c51 Jul 09 18:05:12.387988 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.387956 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-config-volume\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.388146 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.388009 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.388146 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.388063 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.388146 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.388091 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.388146 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.388121 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7678ee9-03ce-4b69-81f1-8101576a1350-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.388359 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.388151 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7678ee9-03ce-4b69-81f1-8101576a1350-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.388359 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.388176 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7678ee9-03ce-4b69-81f1-8101576a1350-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.388359 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.388214 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6js9\" (UniqueName: \"kubernetes.io/projected/e7678ee9-03ce-4b69-81f1-8101576a1350-kube-api-access-z6js9\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.388359 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.388245 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.388359 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.388269 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-web-config\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.388359 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.388296 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.388359 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.388325 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7678ee9-03ce-4b69-81f1-8101576a1350-config-out\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.388359 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:05:12.388346 2569 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 09 18:05:12.388359 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.388358 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e7678ee9-03ce-4b69-81f1-8101576a1350-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.388821 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:05:12.388413 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-main-tls podName:e7678ee9-03ce-4b69-81f1-8101576a1350 nodeName:}" failed. No retries permitted until 2026-07-09 18:05:12.888391207 +0000 UTC m=+69.353913597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "e7678ee9-03ce-4b69-81f1-8101576a1350") : secret "alertmanager-main-tls" not found Jul 09 18:05:12.388821 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.388729 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e7678ee9-03ce-4b69-81f1-8101576a1350-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.389228 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.389200 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7678ee9-03ce-4b69-81f1-8101576a1350-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.389901 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.389878 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7678ee9-03ce-4b69-81f1-8101576a1350-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.391731 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.391709 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-config-volume\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.400303 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.399875 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6js9\" (UniqueName: \"kubernetes.io/projected/e7678ee9-03ce-4b69-81f1-8101576a1350-kube-api-access-z6js9\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.400303 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.400266 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7678ee9-03ce-4b69-81f1-8101576a1350-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.400718 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.400629 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7678ee9-03ce-4b69-81f1-8101576a1350-config-out\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.401038 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.400929 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.401038 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.400929 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-web-config\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.401038 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.400996 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.401215 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.401050 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.402166 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.402144 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.555120 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.555075 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cr5qg" event={"ID":"25941c59-99b0-40e3-98fd-687a15e8da9e","Type":"ContainerStarted","Data":"7f3ee2ccc520cd263a558412479d4a080e78322ed9a4a34202312e12c52cebfc"} Jul 09 18:05:12.555559 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.555153 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cr5qg" event={"ID":"25941c59-99b0-40e3-98fd-687a15e8da9e","Type":"ContainerStarted","Data":"62d23ce6f0fab91cd706c4f919dbcde9667202714c744125579fb2074b37d327"} Jul 09 18:05:12.557631 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.557600 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dszrz" event={"ID":"bbdd2842-6040-47e2-964f-613c8026aebf","Type":"ContainerStarted","Data":"d8e558cf69aa2d88cce6a34ede51195103b944fa3b41a65e60ca565241ebf998"} Jul 09 18:05:12.560262 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.560232 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" event={"ID":"37199f74-3a3d-4d96-9b8d-7c248d641d11","Type":"ContainerStarted","Data":"bcab5de0617b265f2ed6a8775c10245930651a5e84e9c6eaf1b65df9ba7c1044"} Jul 09 18:05:12.562128 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.562102 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" event={"ID":"05a00921-ffc8-419d-b9e1-e76d8c6580ee","Type":"ContainerStarted","Data":"5187792192bfcc1cdb899141b6f5a16d8e14bbc73ff6c1a766ba8fcddc8858ae"} Jul 09 18:05:12.562240 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.562197 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" event={"ID":"05a00921-ffc8-419d-b9e1-e76d8c6580ee","Type":"ContainerStarted","Data":"f9c905bfbb00998dd5e4d7e05097ba8e66e42e806f905f9bcd7309a4b1aa7c51"} Jul 09 18:05:12.576920 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.576129 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cr5qg" podStartSLOduration=67.065138281 podStartE2EDuration="1m8.576112327s" totalImagesPullingTime="1.510974046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:12.575957464 +0000 UTC m=+69.041479888" watchObservedRunningTime="2026-07-09 18:05:12.576112327 +0000 UTC m=+69.041634738" Jul 09 18:05:12.713182 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.713084 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:12.814062 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.814031 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:12.892518 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.892362 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:12.897445 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:12.897363 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:13.194310 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.194276 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:05:13.234721 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.234626 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-68c4666c44-5grzf"] Jul 09 18:05:13.238354 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.237414 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.241814 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.241079 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 09 18:05:13.241814 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.241132 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 09 18:05:13.241814 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.241278 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 09 18:05:13.241814 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.241293 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 09 18:05:13.241814 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.241369 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2xcp6\"" Jul 09 18:05:13.241814 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.241091 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5u77lr52g8i3u\"" Jul 09 18:05:13.241814 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.241632 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 09 18:05:13.258504 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.258473 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-68c4666c44-5grzf"] Jul 09 18:05:13.295512 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.295478 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.295512 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.295526 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.295816 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.295582 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.295816 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.295634 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7h94\" (UniqueName: \"kubernetes.io/projected/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-kube-api-access-q7h94\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.295816 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.295653 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-grpc-tls\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.295816 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.295678 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-tls\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.295816 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.295756 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.295816 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.295779 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-metrics-client-ca\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.396872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.396837 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-tls\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.397049 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.396894 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.397161 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.397134 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-metrics-client-ca\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.397233 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.397216 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.397295 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.397255 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.397405 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.397291 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.397405 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.397337 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7h94\" (UniqueName: \"kubernetes.io/projected/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-kube-api-access-q7h94\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.397405 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.397369 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-grpc-tls\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.398659 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.398631 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-metrics-client-ca\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.400076 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.400048 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.400364 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.400346 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-grpc-tls\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.400438 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.400383 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.401580 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.401554 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.401660 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.401565 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-tls\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.402015 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.401994 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.412793 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.412751 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7h94\" (UniqueName: \"kubernetes.io/projected/c332145d-ab9e-4777-aa6f-4ffa8c0a2de0-kube-api-access-q7h94\") pod \"thanos-querier-68c4666c44-5grzf\" (UID: \"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0\") " pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.550620 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.550523 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:13.568346 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.568303 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dszrz" event={"ID":"bbdd2842-6040-47e2-964f-613c8026aebf","Type":"ContainerStarted","Data":"2337347785ca8a92619f360dd096d7d36f2c30c12445b4feb6d640636244a735"} Jul 09 18:05:13.570570 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.570535 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" event={"ID":"05a00921-ffc8-419d-b9e1-e76d8c6580ee","Type":"ContainerStarted","Data":"b1afe58623e8f1056f697fbfaf481b7075d2482d6cd2d52d3a0358c0b349facc"} Jul 09 18:05:13.624472 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.624432 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:13.741112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:13.741074 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:14.152367 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:14.152330 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:14.279560 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:14.279527 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:14.495031 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:14.494998 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:14.576243 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:14.576209 2569 generic.go:356] "Generic (PLEG): container finished" podID="bbdd2842-6040-47e2-964f-613c8026aebf" containerID="2337347785ca8a92619f360dd096d7d36f2c30c12445b4feb6d640636244a735" exitCode=0 Jul 09 18:05:14.576731 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:14.576274 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dszrz" event={"ID":"bbdd2842-6040-47e2-964f-613c8026aebf","Type":"ContainerDied","Data":"2337347785ca8a92619f360dd096d7d36f2c30c12445b4feb6d640636244a735"} Jul 09 18:05:15.264680 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.264640 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:15.714558 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.714519 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:05:15.779045 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.779012 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7"] Jul 09 18:05:15.781501 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.781459 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.788452 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.788418 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 09 18:05:15.788602 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.788459 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-jlpx8\"" Jul 09 18:05:15.789386 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.789179 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 09 18:05:15.791092 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.790741 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 09 18:05:15.791092 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.790948 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-anpokgr77cvip\"" Jul 09 18:05:15.796776 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.796758 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 18:05:15.818315 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.818146 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d91372df-546b-4b7e-9ee1-c2d8d718b727-metrics-server-audit-profiles\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.818315 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.818195 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d91372df-546b-4b7e-9ee1-c2d8d718b727-audit-log\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.818315 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.818245 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d91372df-546b-4b7e-9ee1-c2d8d718b727-secret-metrics-server-client-certs\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.818558 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.818322 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d91372df-546b-4b7e-9ee1-c2d8d718b727-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.818558 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.818353 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6m56\" (UniqueName: \"kubernetes.io/projected/d91372df-546b-4b7e-9ee1-c2d8d718b727-kube-api-access-m6m56\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.818558 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.818395 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d91372df-546b-4b7e-9ee1-c2d8d718b727-client-ca-bundle\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.818558 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.818492 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d91372df-546b-4b7e-9ee1-c2d8d718b727-secret-metrics-server-tls\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.823164 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.823134 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7"] Jul 09 18:05:15.904396 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:05:15.904361 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7678ee9_03ce_4b69_81f1_8101576a1350.slice/crio-841503f5005226f10df4f9d9c6d31f9337baa750699b4e5547584cfac81a0c45 WatchSource:0}: Error finding container 841503f5005226f10df4f9d9c6d31f9337baa750699b4e5547584cfac81a0c45: Status 404 returned error can't find the container with id 841503f5005226f10df4f9d9c6d31f9337baa750699b4e5547584cfac81a0c45 Jul 09 18:05:15.914105 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.914079 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt"] Jul 09 18:05:15.916068 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.916037 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:15.916502 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.916476 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt" Jul 09 18:05:15.922539 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.919266 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d91372df-546b-4b7e-9ee1-c2d8d718b727-client-ca-bundle\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.922539 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.919317 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d91372df-546b-4b7e-9ee1-c2d8d718b727-secret-metrics-server-tls\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.922539 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.919386 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d91372df-546b-4b7e-9ee1-c2d8d718b727-metrics-server-audit-profiles\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.922539 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.919412 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d91372df-546b-4b7e-9ee1-c2d8d718b727-audit-log\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.922539 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.919451 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d91372df-546b-4b7e-9ee1-c2d8d718b727-secret-metrics-server-client-certs\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.922539 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.919489 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d91372df-546b-4b7e-9ee1-c2d8d718b727-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.922539 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.919515 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6m56\" (UniqueName: \"kubernetes.io/projected/d91372df-546b-4b7e-9ee1-c2d8d718b727-kube-api-access-m6m56\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.922539 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.920340 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d91372df-546b-4b7e-9ee1-c2d8d718b727-audit-log\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.922539 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.921856 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 09 18:05:15.922539 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.922100 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rb5k8\"" Jul 09 18:05:15.923407 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.922756 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d91372df-546b-4b7e-9ee1-c2d8d718b727-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.923407 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.923081 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d91372df-546b-4b7e-9ee1-c2d8d718b727-metrics-server-audit-profiles\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.926143 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.926120 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d91372df-546b-4b7e-9ee1-c2d8d718b727-secret-metrics-server-tls\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.927192 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.927012 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d91372df-546b-4b7e-9ee1-c2d8d718b727-secret-metrics-server-client-certs\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.927192 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.927137 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d91372df-546b-4b7e-9ee1-c2d8d718b727-client-ca-bundle\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.938081 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.938053 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6m56\" (UniqueName: \"kubernetes.io/projected/d91372df-546b-4b7e-9ee1-c2d8d718b727-kube-api-access-m6m56\") pod \"metrics-server-7b8b74fc9f-mpjm7\" (UID: \"d91372df-546b-4b7e-9ee1-c2d8d718b727\") " pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:15.944386 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:15.944307 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt"] Jul 09 18:05:16.020187 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.020146 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8l2q\" (UniqueName: \"kubernetes.io/projected/514852e2-de26-4c7f-b071-f875cefae908-kube-api-access-l8l2q\") pod \"monitoring-plugin-5f68df7f58-p8sdt\" (UID: \"514852e2-de26-4c7f-b071-f875cefae908\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt" Jul 09 18:05:16.020373 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.020231 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/514852e2-de26-4c7f-b071-f875cefae908-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-p8sdt\" (UID: \"514852e2-de26-4c7f-b071-f875cefae908\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt" Jul 09 18:05:16.065933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.065898 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-68c4666c44-5grzf"] Jul 09 18:05:16.087547 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.087475 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:16.092309 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.092287 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:16.121668 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.121628 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8l2q\" (UniqueName: \"kubernetes.io/projected/514852e2-de26-4c7f-b071-f875cefae908-kube-api-access-l8l2q\") pod \"monitoring-plugin-5f68df7f58-p8sdt\" (UID: \"514852e2-de26-4c7f-b071-f875cefae908\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt" Jul 09 18:05:16.121838 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.121732 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/514852e2-de26-4c7f-b071-f875cefae908-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-p8sdt\" (UID: \"514852e2-de26-4c7f-b071-f875cefae908\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt" Jul 09 18:05:16.124651 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.124627 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/514852e2-de26-4c7f-b071-f875cefae908-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-p8sdt\" (UID: \"514852e2-de26-4c7f-b071-f875cefae908\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt" Jul 09 18:05:16.133963 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.133939 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8l2q\" (UniqueName: \"kubernetes.io/projected/514852e2-de26-4c7f-b071-f875cefae908-kube-api-access-l8l2q\") pod \"monitoring-plugin-5f68df7f58-p8sdt\" (UID: \"514852e2-de26-4c7f-b071-f875cefae908\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt" Jul 09 18:05:16.152979 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:05:16.152950 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc332145d_ab9e_4777_aa6f_4ffa8c0a2de0.slice/crio-1052c56aec9f5394fd28ae03bfc882dc26ba3e38d5c20acde4cb91875feee52a WatchSource:0}: Error finding container 1052c56aec9f5394fd28ae03bfc882dc26ba3e38d5c20acde4cb91875feee52a: Status 404 returned error can't find the container with id 1052c56aec9f5394fd28ae03bfc882dc26ba3e38d5c20acde4cb91875feee52a Jul 09 18:05:16.231447 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.231188 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt" Jul 09 18:05:16.420734 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.420627 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7"] Jul 09 18:05:16.424948 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:05:16.424906 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd91372df_546b_4b7e_9ee1_c2d8d718b727.slice/crio-8d11814e7351bd2a0134a01e74ea9f68253c5082e022072c339b14f05f32d5ec WatchSource:0}: Error finding container 8d11814e7351bd2a0134a01e74ea9f68253c5082e022072c339b14f05f32d5ec: Status 404 returned error can't find the container with id 8d11814e7351bd2a0134a01e74ea9f68253c5082e022072c339b14f05f32d5ec Jul 09 18:05:16.440399 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.440359 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6d88b99897-xdjwj"] Jul 09 18:05:16.447068 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.446912 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.456633 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.453494 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 09 18:05:16.456633 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.453942 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-g25pp\"" Jul 09 18:05:16.456633 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.454348 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt"] Jul 09 18:05:16.456633 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.454482 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 09 18:05:16.456633 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.455380 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 09 18:05:16.456633 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.456039 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 09 18:05:16.456633 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.456253 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 09 18:05:16.470600 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.470264 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 09 18:05:16.475534 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.473640 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d88b99897-xdjwj"] Jul 09 18:05:16.526414 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.526126 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-federate-client-tls\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.526414 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.526177 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.526414 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.526220 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-serving-certs-ca-bundle\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.526414 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.526267 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-secret-telemeter-client\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.526414 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.526309 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8r8h\" (UniqueName: \"kubernetes.io/projected/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-kube-api-access-j8r8h\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.526414 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.526353 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.526414 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.526388 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-metrics-client-ca\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.526927 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.526424 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-telemeter-client-tls\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.529852 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.529390 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mtt6h" Jul 09 18:05:16.593029 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.592166 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sq92v" event={"ID":"e1661ba5-6bd9-484b-97a0-8b28cb87e456","Type":"ContainerStarted","Data":"3d8ead7fd2eb22c18114214eb72b7a0498e62422f5bf6ef101f130008712490e"} Jul 09 18:05:16.605444 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.604739 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dszrz" event={"ID":"bbdd2842-6040-47e2-964f-613c8026aebf","Type":"ContainerStarted","Data":"bef2ce962de3c432fcc8613c2b6639a174c75469b499e0efa75e532dae6392ec"} Jul 09 18:05:16.605444 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.604802 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dszrz" event={"ID":"bbdd2842-6040-47e2-964f-613c8026aebf","Type":"ContainerStarted","Data":"e21d66400d4083085f23e900f22c2dd58eacf026d47531f2f89828fbe86b952f"} Jul 09 18:05:16.609737 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.609179 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" event={"ID":"37199f74-3a3d-4d96-9b8d-7c248d641d11","Type":"ContainerStarted","Data":"7361beb1bc1f9d220e70c5a801b856ffa7b2bd9afb41b5f3b09dfb5d914dbab9"} Jul 09 18:05:16.609737 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.609232 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" event={"ID":"37199f74-3a3d-4d96-9b8d-7c248d641d11","Type":"ContainerStarted","Data":"4590c930c492187418a9e399be7ea40068fa8ee9a354d55158a161accbb6eb10"} Jul 09 18:05:16.609737 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.609252 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" event={"ID":"37199f74-3a3d-4d96-9b8d-7c248d641d11","Type":"ContainerStarted","Data":"3f85517736572f56d66a9e511f0e235ebbfcc5ae8b5da1c20fc134a3de070fcf"} Jul 09 18:05:16.611952 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.611383 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerStarted","Data":"841503f5005226f10df4f9d9c6d31f9337baa750699b4e5547584cfac81a0c45"} Jul 09 18:05:16.614112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.613952 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sq92v" podStartSLOduration=66.500601855 podStartE2EDuration="1m12.613931523s" totalImagesPullingTime="6.113329668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:16.612303112 +0000 UTC m=+73.077825521" watchObservedRunningTime="2026-07-09 18:05:16.613931523 +0000 UTC m=+73.079453937" Jul 09 18:05:16.614771 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.614749 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lntgg" event={"ID":"aa5fad78-82ce-49bb-a50e-2b178586bdaf","Type":"ContainerStarted","Data":"624f901a8b7613e20540fa6b2788289c8795cae016037262467a9b532854cbcd"} Jul 09 18:05:16.614911 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.614898 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:05:16.617826 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.617805 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" event={"ID":"05a00921-ffc8-419d-b9e1-e76d8c6580ee","Type":"ContainerStarted","Data":"474f74939ad87058cc4e7fcfed6d306b7f560042a7b247d70b5e4c703c1fc81c"} Jul 09 18:05:16.619197 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.619169 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt" event={"ID":"514852e2-de26-4c7f-b071-f875cefae908","Type":"ContainerStarted","Data":"e11e7899113e0b93d09f7298155b4c1488e8f7799aa829be0f0d917c4b128dbb"} Jul 09 18:05:16.621150 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.621128 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" event={"ID":"d91372df-546b-4b7e-9ee1-c2d8d718b727","Type":"ContainerStarted","Data":"8d11814e7351bd2a0134a01e74ea9f68253c5082e022072c339b14f05f32d5ec"} Jul 09 18:05:16.623320 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.623244 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" event={"ID":"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0","Type":"ContainerStarted","Data":"1052c56aec9f5394fd28ae03bfc882dc26ba3e38d5c20acde4cb91875feee52a"} Jul 09 18:05:16.628049 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.626905 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8r8h\" (UniqueName: \"kubernetes.io/projected/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-kube-api-access-j8r8h\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.628049 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.627049 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.628049 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.627116 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-metrics-client-ca\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.628049 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.627151 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-telemeter-client-tls\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.628049 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.627265 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-federate-client-tls\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.628049 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.627295 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.628049 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.627372 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-serving-certs-ca-bundle\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.628049 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.627546 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-secret-telemeter-client\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.630946 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.629916 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-serving-certs-ca-bundle\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.630946 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.630206 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.631744 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.631309 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-metrics-client-ca\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.632553 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.632450 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-secret-telemeter-client\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.634377 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.633732 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-7lqhs" podStartSLOduration=1.238180257 podStartE2EDuration="5.633714445s" totalImagesPullingTime="4.395534188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:16.63156976 +0000 UTC m=+73.097092172" watchObservedRunningTime="2026-07-09 18:05:16.633714445 +0000 UTC m=+73.099236858" Jul 09 18:05:16.634479 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.634400 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-federate-client-tls\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.634966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.634923 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.635619 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.635596 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-telemeter-client-tls\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.638944 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.638902 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8r8h\" (UniqueName: \"kubernetes.io/projected/5bff25a4-76cb-4bb4-92d9-b31ca66ffd71-kube-api-access-j8r8h\") pod \"telemeter-client-6d88b99897-xdjwj\" (UID: \"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71\") " pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.664094 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.664033 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dszrz" podStartSLOduration=4.348943782 podStartE2EDuration="5.664017598s" totalImagesPullingTime="1.219733747s" totalInitContainerRuntime="95.340069ms" isStatefulPod=true podCreationTimestamp="2026-07-09 18:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:16.663250519 +0000 UTC m=+73.128772930" watchObservedRunningTime="2026-07-09 18:05:16.664017598 +0000 UTC m=+73.129540010" Jul 09 18:05:16.682928 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.682874 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lntgg" podStartSLOduration=66.874779951 podStartE2EDuration="1m12.682857032s" totalImagesPullingTime="5.808077081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:16.68204018 +0000 UTC m=+73.147562616" watchObservedRunningTime="2026-07-09 18:05:16.682857032 +0000 UTC m=+73.148379444" Jul 09 18:05:16.718107 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.718034 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-brlv8" podStartSLOduration=2.408367194 podStartE2EDuration="5.718017093s" totalImagesPullingTime="3.309649899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:16.717134741 +0000 UTC m=+73.182657153" watchObservedRunningTime="2026-07-09 18:05:16.718017093 +0000 UTC m=+73.183539504" Jul 09 18:05:16.770787 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.770746 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" Jul 09 18:05:16.948239 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:16.948075 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d88b99897-xdjwj"] Jul 09 18:05:17.202417 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:05:17.202344 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bff25a4_76cb_4bb4_92d9_b31ca66ffd71.slice/crio-76a111f3f2f2b916ae3de3f40c49c6dbb5790ea2fa5694e1b42a701210006b03 WatchSource:0}: Error finding container 76a111f3f2f2b916ae3de3f40c49c6dbb5790ea2fa5694e1b42a701210006b03: Status 404 returned error can't find the container with id 76a111f3f2f2b916ae3de3f40c49c6dbb5790ea2fa5694e1b42a701210006b03 Jul 09 18:05:17.529660 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.529567 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:17.596000 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.593787 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:05:17.601985 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.599496 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.603626 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.603595 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 18:05:17.603985 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.603966 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 18:05:17.604264 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.604244 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 18:05:17.604513 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.604497 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5is2hrefkin27\"" Jul 09 18:05:17.605614 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.604708 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 18:05:17.605614 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.604894 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 18:05:17.606301 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.605931 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 18:05:17.606301 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.606130 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 18:05:17.606746 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.606725 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 18:05:17.606994 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.606904 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 18:05:17.611214 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.611176 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 18:05:17.611325 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.611308 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 18:05:17.613798 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.612760 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 18:05:17.613798 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.613041 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 18:05:17.613798 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.613330 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-w7cqr\"" Jul 09 18:05:17.622927 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.622899 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 18:05:17.638243 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.637503 2569 generic.go:356] "Generic (PLEG): container finished" podID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerID="a304497ab8668f0c7f0d39592dab1a6a95fce6ebe3559b00fe3dd82277822b0c" exitCode=0 Jul 09 18:05:17.638243 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.637717 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerDied","Data":"a304497ab8668f0c7f0d39592dab1a6a95fce6ebe3559b00fe3dd82277822b0c"} Jul 09 18:05:17.638243 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.637996 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-web-config\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.638243 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.638036 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.638243 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.638064 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.638243 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.638100 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.638243 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.638162 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-config\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.638243 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.638199 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.638961 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.638815 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39e0cb81-24ee-4743-903e-3d1114e0c8e7-config-out\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.638961 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.638864 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.638961 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.638899 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39e0cb81-24ee-4743-903e-3d1114e0c8e7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.638961 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.638923 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.639218 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.638979 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.639218 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.639023 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.639218 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.639050 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.639218 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.639077 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.639218 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.639121 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.639218 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.639152 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.639218 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.639192 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.639218 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.639216 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.639835 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.639280 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqcxm\" (UniqueName: \"kubernetes.io/projected/39e0cb81-24ee-4743-903e-3d1114e0c8e7-kube-api-access-cqcxm\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.639835 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.639313 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.640113 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.639991 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" event={"ID":"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71","Type":"ContainerStarted","Data":"76a111f3f2f2b916ae3de3f40c49c6dbb5790ea2fa5694e1b42a701210006b03"} Jul 09 18:05:17.740137 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.740021 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-config\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.740137 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.740069 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.740658 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.740544 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39e0cb81-24ee-4743-903e-3d1114e0c8e7-config-out\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.740658 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.740581 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.740892 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.740869 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39e0cb81-24ee-4743-903e-3d1114e0c8e7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.740961 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.740907 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.741032 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.741013 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.741536 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.741330 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.741649 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.741555 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.741649 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.741618 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.741796 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.741655 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.741796 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.741668 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.741796 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.741701 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.741796 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.741769 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.741796 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.741781 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.742035 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.741798 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.742035 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.741840 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.742035 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.741864 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.742035 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.741914 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqcxm\" (UniqueName: \"kubernetes.io/projected/39e0cb81-24ee-4743-903e-3d1114e0c8e7-kube-api-access-cqcxm\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.742035 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.741968 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.742270 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.742099 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-web-config\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.742270 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.742133 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.742270 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.742186 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.742414 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.742269 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.742469 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.742436 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.742963 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.742934 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.743238 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.743183 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.744202 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.743595 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-config\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.749388 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.749363 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.750166 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.750144 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.751231 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.751188 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.756558 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.756459 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.756558 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.756468 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.756954 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.756930 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.758620 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.758195 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39e0cb81-24ee-4743-903e-3d1114e0c8e7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.758620 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.758304 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.758620 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.758575 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-web-config\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.759508 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.759219 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.759800 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.759677 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39e0cb81-24ee-4743-903e-3d1114e0c8e7-config-out\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.768055 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.767975 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:05:17.770230 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.770153 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqcxm\" (UniqueName: \"kubernetes.io/projected/39e0cb81-24ee-4743-903e-3d1114e0c8e7-kube-api-access-cqcxm\") pod \"prometheus-k8s-0\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:17.770991 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.770815 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:17.917960 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:17.917849 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:18.509918 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:18.509880 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:18.610886 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:18.610790 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:18.732021 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:18.731893 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:05:18.734908 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:05:18.734875 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39e0cb81_24ee_4743_903e_3d1114e0c8e7.slice/crio-9fcdf59752ce15bd219e2f09f329c0504376e273aab3081c01c8c40fa9e6c74e WatchSource:0}: Error finding container 9fcdf59752ce15bd219e2f09f329c0504376e273aab3081c01c8c40fa9e6c74e: Status 404 returned error can't find the container with id 9fcdf59752ce15bd219e2f09f329c0504376e273aab3081c01c8c40fa9e6c74e Jul 09 18:05:18.840496 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:18.840472 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:18.949215 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:18.949182 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:19.660418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:19.660381 2569 generic.go:356] "Generic (PLEG): container finished" podID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerID="26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4" exitCode=0 Jul 09 18:05:19.660616 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:19.660495 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerDied","Data":"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4"} Jul 09 18:05:19.660616 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:19.660601 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerStarted","Data":"9fcdf59752ce15bd219e2f09f329c0504376e273aab3081c01c8c40fa9e6c74e"} Jul 09 18:05:19.663295 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:19.663263 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt" event={"ID":"514852e2-de26-4c7f-b071-f875cefae908","Type":"ContainerStarted","Data":"4d371fe2adcb0aa9ecee90e30689a09bacc3bb999db51abf6618bcd13ddc5d89"} Jul 09 18:05:19.664250 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:19.664231 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt" Jul 09 18:05:19.667626 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:19.667582 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" event={"ID":"d91372df-546b-4b7e-9ee1-c2d8d718b727","Type":"ContainerStarted","Data":"dc75ef01d5031e64ce04aeac2d6d8d6183877224380744df4d3c9177465849d3"} Jul 09 18:05:19.670123 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:19.670079 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt" Jul 09 18:05:19.721515 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:19.719939 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" podStartSLOduration=2.606902822 podStartE2EDuration="4.719920504s" totalImagesPullingTime="2.113017682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:05:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:19.719397831 +0000 UTC m=+76.184920242" watchObservedRunningTime="2026-07-09 18:05:19.719920504 +0000 UTC m=+76.185442913" Jul 09 18:05:19.742071 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:19.742011 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p8sdt" podStartSLOduration=2.669834963 podStartE2EDuration="4.741987631s" totalImagesPullingTime="2.072152668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:05:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:19.740843504 +0000 UTC m=+76.206365934" watchObservedRunningTime="2026-07-09 18:05:19.741987631 +0000 UTC m=+76.207510045" Jul 09 18:05:19.949723 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:19.949620 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:20.037086 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:20.037051 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:21.683840 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:21.683794 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerStarted","Data":"06b6f532105bfebefe24a2fccec3e622628494825a40d0e4515f073817fb59cc"} Jul 09 18:05:21.684267 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:21.683919 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerStarted","Data":"45f0629b0ba4067f858be958cc6abe5d9d0edb5533141e9279bd0e2378fca0e1"} Jul 09 18:05:21.684267 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:21.683966 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerStarted","Data":"8af829a1248c53ccf8ae527ffbe500988203238fd274bcd67e0652a927e780f8"} Jul 09 18:05:21.684267 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:21.684008 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerStarted","Data":"a6718bf11c30b7a9fa159a35badc68b5b0cb324d152215d7c413212ca860d1d7"} Jul 09 18:05:21.684267 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:21.684047 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerStarted","Data":"2406ca5d4ac85dd8a3a10fc055e301f1357d2aba424d9ea645ff5e510c3517ae"} Jul 09 18:05:21.686629 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:21.686599 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" event={"ID":"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71","Type":"ContainerStarted","Data":"a721456173453094a2ffbffcf9464b8d64c73b402e16c4d657c4fc7582ad3ba5"} Jul 09 18:05:21.686802 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:21.686662 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" event={"ID":"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71","Type":"ContainerStarted","Data":"2200594bc20232450039e1557e87afcb2ac69b3278b55513a45ee5396345625e"} Jul 09 18:05:21.686802 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:21.686711 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" event={"ID":"5bff25a4-76cb-4bb4-92d9-b31ca66ffd71","Type":"ContainerStarted","Data":"ae5fdb1091370084fa57355ad9e54821116a2eb4c682e4da793727b224c539d9"} Jul 09 18:05:21.689300 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:21.689269 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" event={"ID":"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0","Type":"ContainerStarted","Data":"e511b787a5f0ade948c2ceb6d199ffb03b1f64d1395c945676078052c88ec4ed"} Jul 09 18:05:21.689432 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:21.689348 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" event={"ID":"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0","Type":"ContainerStarted","Data":"35b8337141d14d4789121cc80ad8e1d5106ff343b835c9bb413881adb35d60c0"} Jul 09 18:05:21.689432 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:21.689390 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" event={"ID":"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0","Type":"ContainerStarted","Data":"de1518668c8c863488571a002f5cb27d8876b635b7399ec8adfe2f0203419466"} Jul 09 18:05:22.130629 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:22.130591 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:22.264362 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:22.264328 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:22.704827 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:22.700909 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerStarted","Data":"cef8076286ce61103eeba101e444821a22afdd9e011fac401d33266765209200"} Jul 09 18:05:22.709850 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:22.709821 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" event={"ID":"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0","Type":"ContainerStarted","Data":"5b85f22c15ed6667fa2b773489277947f56f6bd597154033a801a8787291eb1c"} Jul 09 18:05:22.710004 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:22.709929 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" event={"ID":"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0","Type":"ContainerStarted","Data":"b356c9f59774ab4cdbc8e1677deeba000d38e332dfa2ec3b1b1db303eee8e600"} Jul 09 18:05:22.710004 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:22.709987 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" event={"ID":"c332145d-ab9e-4777-aa6f-4ffa8c0a2de0","Type":"ContainerStarted","Data":"3e381af5dbf61d16c770bdefcfa2507e4854ebb10fc100f54334d25e9f70e37e"} Jul 09 18:05:22.752315 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:22.752217 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.131220967 podStartE2EDuration="10.752194334s" totalImagesPullingTime="5.596327207s" totalInitContainerRuntime="24.64616ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:05:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:22.748124477 +0000 UTC m=+79.213646888" watchObservedRunningTime="2026-07-09 18:05:22.752194334 +0000 UTC m=+79.217716748" Jul 09 18:05:22.752515 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:22.752488 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6d88b99897-xdjwj" podStartSLOduration=3.008781581 podStartE2EDuration="6.75243572s" totalImagesPullingTime="3.743654139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:05:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:21.722641265 +0000 UTC m=+78.188163676" watchObservedRunningTime="2026-07-09 18:05:22.75243572 +0000 UTC m=+79.217958132" Jul 09 18:05:22.785071 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:22.785008 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" podStartSLOduration=3.898957555 podStartE2EDuration="9.784985191s" totalImagesPullingTime="5.886027636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:05:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:22.782076492 +0000 UTC m=+79.247598925" watchObservedRunningTime="2026-07-09 18:05:22.784985191 +0000 UTC m=+79.250507603" Jul 09 18:05:23.714875 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:23.714705 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:24.726698 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:24.726642 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerStarted","Data":"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549"} Jul 09 18:05:24.727045 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:24.726763 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerStarted","Data":"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6"} Jul 09 18:05:24.727045 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:24.726791 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerStarted","Data":"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630"} Jul 09 18:05:24.727045 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:24.726816 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerStarted","Data":"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c"} Jul 09 18:05:24.727045 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:24.726845 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerStarted","Data":"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff"} Jul 09 18:05:24.727045 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:24.726867 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerStarted","Data":"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb"} Jul 09 18:05:24.772974 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:24.772920 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.166983991 podStartE2EDuration="7.77290054s" totalImagesPullingTime="4.577647696s" totalInitContainerRuntime="28.268853ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:05:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:05:24.769291184 +0000 UTC m=+81.234813605" watchObservedRunningTime="2026-07-09 18:05:24.77290054 +0000 UTC m=+81.238422950" Jul 09 18:05:25.186471 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:25.186387 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:25.341264 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:25.341230 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:25.701675 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:25.701644 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:25.804590 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:25.804557 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:25.885258 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:25.885216 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:27.416450 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:27.416418 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:27.584418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:27.584385 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:27.918128 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:27.918090 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:05:29.733045 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:29.733015 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-68c4666c44-5grzf" Jul 09 18:05:29.997291 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:29.997199 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:30.136110 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:30.136078 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:32.637294 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:32.637261 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:32.739659 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:32.739632 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:32.839220 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:32.839186 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:35.480165 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:35.480135 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:35.544260 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:35.544228 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:36.093226 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:36.093195 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:36.093410 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:36.093247 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:40.068818 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:40.068770 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:40.183069 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:40.183036 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:40.836049 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:40.836014 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:41.824980 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:41.824946 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:47.643143 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:47.643111 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lntgg" Jul 09 18:05:47.691561 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:47.691527 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:47.793411 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:47.793371 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:50.295152 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:50.295127 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7678ee9-03ce-4b69-81f1-8101576a1350/init-config-reloader/0.log" Jul 09 18:05:50.302205 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:50.302186 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7678ee9-03ce-4b69-81f1-8101576a1350/alertmanager/0.log" Jul 09 18:05:50.396646 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:50.396625 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7678ee9-03ce-4b69-81f1-8101576a1350/config-reloader/0.log" Jul 09 18:05:50.594166 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:50.594091 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7678ee9-03ce-4b69-81f1-8101576a1350/kube-rbac-proxy-web/0.log" Jul 09 18:05:50.793828 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:50.793800 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7678ee9-03ce-4b69-81f1-8101576a1350/kube-rbac-proxy/0.log" Jul 09 18:05:50.994187 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:50.994113 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7678ee9-03ce-4b69-81f1-8101576a1350/kube-rbac-proxy-metric/0.log" Jul 09 18:05:51.193306 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:51.193283 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7678ee9-03ce-4b69-81f1-8101576a1350/prom-label-proxy/0.log" Jul 09 18:05:51.593524 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:51.593501 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-7lqhs_37199f74-3a3d-4d96-9b8d-7c248d641d11/kube-state-metrics/0.log" Jul 09 18:05:51.794034 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:51.793983 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-7lqhs_37199f74-3a3d-4d96-9b8d-7c248d641d11/kube-rbac-proxy-main/0.log" Jul 09 18:05:51.876657 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:51.876571 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:51.993367 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:51.993313 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-7lqhs_37199f74-3a3d-4d96-9b8d-7c248d641d11/kube-rbac-proxy-self/0.log" Jul 09 18:05:52.037045 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:52.037005 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:52.193404 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:52.193324 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7b8b74fc9f-mpjm7_d91372df-546b-4b7e-9ee1-c2d8d718b727/metrics-server/0.log" Jul 09 18:05:52.346940 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:52.346904 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:52.392802 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:52.392781 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-p8sdt_514852e2-de26-4c7f-b071-f875cefae908/monitoring-plugin/0.log" Jul 09 18:05:52.447111 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:52.447043 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:52.548959 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:52.548928 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:52.593623 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:52.593603 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dszrz_bbdd2842-6040-47e2-964f-613c8026aebf/init-textfile/0.log" Jul 09 18:05:52.633512 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:52.633478 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:52.794226 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:52.794200 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dszrz_bbdd2842-6040-47e2-964f-613c8026aebf/node-exporter/0.log" Jul 09 18:05:52.993850 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:52.993825 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dszrz_bbdd2842-6040-47e2-964f-613c8026aebf/kube-rbac-proxy/0.log" Jul 09 18:05:54.393799 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:54.393771 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-brlv8_05a00921-ffc8-419d-b9e1-e76d8c6580ee/kube-rbac-proxy-main/0.log" Jul 09 18:05:54.593867 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:54.593844 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-brlv8_05a00921-ffc8-419d-b9e1-e76d8c6580ee/kube-rbac-proxy-self/0.log" Jul 09 18:05:54.793948 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:54.793919 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-brlv8_05a00921-ffc8-419d-b9e1-e76d8c6580ee/openshift-state-metrics/0.log" Jul 09 18:05:54.997096 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:54.997067 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_39e0cb81-24ee-4743-903e-3d1114e0c8e7/init-config-reloader/0.log" Jul 09 18:05:55.198395 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:55.198322 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_39e0cb81-24ee-4743-903e-3d1114e0c8e7/prometheus/0.log" Jul 09 18:05:55.394425 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:55.394394 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_39e0cb81-24ee-4743-903e-3d1114e0c8e7/config-reloader/0.log" Jul 09 18:05:55.594858 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:55.594832 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_39e0cb81-24ee-4743-903e-3d1114e0c8e7/thanos-sidecar/0.log" Jul 09 18:05:55.795446 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:55.795415 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_39e0cb81-24ee-4743-903e-3d1114e0c8e7/kube-rbac-proxy-web/0.log" Jul 09 18:05:55.887145 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:55.887070 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:55.999350 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:55.999317 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_39e0cb81-24ee-4743-903e-3d1114e0c8e7/kube-rbac-proxy/0.log" Jul 09 18:05:56.013783 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:56.013755 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:05:56.098567 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:56.098542 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:56.102540 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:56.102516 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7b8b74fc9f-mpjm7" Jul 09 18:05:56.194442 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:56.194369 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_39e0cb81-24ee-4743-903e-3d1114e0c8e7/kube-rbac-proxy-thanos/0.log" Jul 09 18:05:56.398803 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:56.398773 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-pkjvw_53f1ad17-1807-49b3-87b1-148ea3b3c791/prometheus-operator/0.log" Jul 09 18:05:56.594935 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:56.594907 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-pkjvw_53f1ad17-1807-49b3-87b1-148ea3b3c791/kube-rbac-proxy/0.log" Jul 09 18:05:56.994500 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:56.994419 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d88b99897-xdjwj_5bff25a4-76cb-4bb4-92d9-b31ca66ffd71/telemeter-client/0.log" Jul 09 18:05:57.193356 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:57.193327 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d88b99897-xdjwj_5bff25a4-76cb-4bb4-92d9-b31ca66ffd71/reload/0.log" Jul 09 18:05:57.394225 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:57.394192 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d88b99897-xdjwj_5bff25a4-76cb-4bb4-92d9-b31ca66ffd71/kube-rbac-proxy/0.log" Jul 09 18:05:57.594910 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:57.594881 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68c4666c44-5grzf_c332145d-ab9e-4777-aa6f-4ffa8c0a2de0/thanos-query/0.log" Jul 09 18:05:57.796044 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:57.796019 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68c4666c44-5grzf_c332145d-ab9e-4777-aa6f-4ffa8c0a2de0/kube-rbac-proxy-web/0.log" Jul 09 18:05:57.993887 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:57.993862 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68c4666c44-5grzf_c332145d-ab9e-4777-aa6f-4ffa8c0a2de0/kube-rbac-proxy/0.log" Jul 09 18:05:58.193705 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:58.193608 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68c4666c44-5grzf_c332145d-ab9e-4777-aa6f-4ffa8c0a2de0/prom-label-proxy/0.log" Jul 09 18:05:58.398114 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:58.398083 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68c4666c44-5grzf_c332145d-ab9e-4777-aa6f-4ffa8c0a2de0/kube-rbac-proxy-rules/0.log" Jul 09 18:05:58.593222 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:05:58.593193 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68c4666c44-5grzf_c332145d-ab9e-4777-aa6f-4ffa8c0a2de0/kube-rbac-proxy-metrics/0.log" Jul 09 18:06:10.824161 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:10.824123 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:06:10.891091 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:10.891056 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:06:17.918378 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:17.918341 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:17.939576 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:17.939550 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:17.965563 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:17.965542 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:26.325151 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:26.325112 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:06:26.426839 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:26.426809 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:06:27.466415 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:27.466382 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:06:27.551825 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:27.551792 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:06:31.825908 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:31.825873 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:06:31.826269 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:31.826201 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="alertmanager" containerID="cri-o://2406ca5d4ac85dd8a3a10fc055e301f1357d2aba424d9ea645ff5e510c3517ae" gracePeriod=120 Jul 09 18:06:31.826269 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:31.826230 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="kube-rbac-proxy-metric" containerID="cri-o://06b6f532105bfebefe24a2fccec3e622628494825a40d0e4515f073817fb59cc" gracePeriod=120 Jul 09 18:06:31.826375 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:31.826286 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="kube-rbac-proxy" containerID="cri-o://45f0629b0ba4067f858be958cc6abe5d9d0edb5533141e9279bd0e2378fca0e1" gracePeriod=120 Jul 09 18:06:31.826431 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:31.826273 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="kube-rbac-proxy-web" containerID="cri-o://8af829a1248c53ccf8ae527ffbe500988203238fd274bcd67e0652a927e780f8" gracePeriod=120 Jul 09 18:06:31.826481 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:31.826280 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="config-reloader" containerID="cri-o://a6718bf11c30b7a9fa159a35badc68b5b0cb324d152215d7c413212ca860d1d7" gracePeriod=120 Jul 09 18:06:31.826481 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:31.826303 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="prom-label-proxy" containerID="cri-o://cef8076286ce61103eeba101e444821a22afdd9e011fac401d33266765209200" gracePeriod=120 Jul 09 18:06:32.013042 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:32.013014 2569 generic.go:356] "Generic (PLEG): container finished" podID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerID="cef8076286ce61103eeba101e444821a22afdd9e011fac401d33266765209200" exitCode=0 Jul 09 18:06:32.013042 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:32.013037 2569 generic.go:356] "Generic (PLEG): container finished" podID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerID="06b6f532105bfebefe24a2fccec3e622628494825a40d0e4515f073817fb59cc" exitCode=0 Jul 09 18:06:32.013042 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:32.013044 2569 generic.go:356] "Generic (PLEG): container finished" podID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerID="45f0629b0ba4067f858be958cc6abe5d9d0edb5533141e9279bd0e2378fca0e1" exitCode=0 Jul 09 18:06:32.013275 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:32.013053 2569 generic.go:356] "Generic (PLEG): container finished" podID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerID="a6718bf11c30b7a9fa159a35badc68b5b0cb324d152215d7c413212ca860d1d7" exitCode=0 Jul 09 18:06:32.013275 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:32.013061 2569 generic.go:356] "Generic (PLEG): container finished" podID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerID="2406ca5d4ac85dd8a3a10fc055e301f1357d2aba424d9ea645ff5e510c3517ae" exitCode=0 Jul 09 18:06:32.013275 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:32.013092 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerDied","Data":"cef8076286ce61103eeba101e444821a22afdd9e011fac401d33266765209200"} Jul 09 18:06:32.013275 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:32.013182 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerDied","Data":"06b6f532105bfebefe24a2fccec3e622628494825a40d0e4515f073817fb59cc"} Jul 09 18:06:32.013275 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:32.013227 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerDied","Data":"45f0629b0ba4067f858be958cc6abe5d9d0edb5533141e9279bd0e2378fca0e1"} Jul 09 18:06:32.013275 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:32.013271 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerDied","Data":"a6718bf11c30b7a9fa159a35badc68b5b0cb324d152215d7c413212ca860d1d7"} Jul 09 18:06:32.013480 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:32.013311 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerDied","Data":"2406ca5d4ac85dd8a3a10fc055e301f1357d2aba424d9ea645ff5e510c3517ae"} Jul 09 18:06:33.021601 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.021572 2569 generic.go:356] "Generic (PLEG): container finished" podID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerID="8af829a1248c53ccf8ae527ffbe500988203238fd274bcd67e0652a927e780f8" exitCode=0 Jul 09 18:06:33.022036 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.021646 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerDied","Data":"8af829a1248c53ccf8ae527ffbe500988203238fd274bcd67e0652a927e780f8"} Jul 09 18:06:33.077335 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.077311 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:33.212718 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.212622 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7678ee9-03ce-4b69-81f1-8101576a1350-tls-assets\") pod \"e7678ee9-03ce-4b69-81f1-8101576a1350\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " Jul 09 18:06:33.212718 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.212673 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy-web\") pod \"e7678ee9-03ce-4b69-81f1-8101576a1350\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " Jul 09 18:06:33.212933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.212731 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e7678ee9-03ce-4b69-81f1-8101576a1350-alertmanager-main-db\") pod \"e7678ee9-03ce-4b69-81f1-8101576a1350\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " Jul 09 18:06:33.212933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.212765 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-main-tls\") pod \"e7678ee9-03ce-4b69-81f1-8101576a1350\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " Jul 09 18:06:33.212933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.212794 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7678ee9-03ce-4b69-81f1-8101576a1350-config-out\") pod \"e7678ee9-03ce-4b69-81f1-8101576a1350\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " Jul 09 18:06:33.212933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.212818 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7678ee9-03ce-4b69-81f1-8101576a1350-alertmanager-trusted-ca-bundle\") pod \"e7678ee9-03ce-4b69-81f1-8101576a1350\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " Jul 09 18:06:33.212933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.212878 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6js9\" (UniqueName: \"kubernetes.io/projected/e7678ee9-03ce-4b69-81f1-8101576a1350-kube-api-access-z6js9\") pod \"e7678ee9-03ce-4b69-81f1-8101576a1350\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " Jul 09 18:06:33.212933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.212906 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7678ee9-03ce-4b69-81f1-8101576a1350-metrics-client-ca\") pod \"e7678ee9-03ce-4b69-81f1-8101576a1350\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " Jul 09 18:06:33.213237 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.212947 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-cluster-tls-config\") pod \"e7678ee9-03ce-4b69-81f1-8101576a1350\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " Jul 09 18:06:33.213237 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.212980 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy\") pod \"e7678ee9-03ce-4b69-81f1-8101576a1350\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " Jul 09 18:06:33.213237 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.213022 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy-metric\") pod \"e7678ee9-03ce-4b69-81f1-8101576a1350\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " Jul 09 18:06:33.213237 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.213046 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-config-volume\") pod \"e7678ee9-03ce-4b69-81f1-8101576a1350\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " Jul 09 18:06:33.213237 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.213085 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-web-config\") pod \"e7678ee9-03ce-4b69-81f1-8101576a1350\" (UID: \"e7678ee9-03ce-4b69-81f1-8101576a1350\") " Jul 09 18:06:33.213237 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.213119 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7678ee9-03ce-4b69-81f1-8101576a1350-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "e7678ee9-03ce-4b69-81f1-8101576a1350" (UID: "e7678ee9-03ce-4b69-81f1-8101576a1350"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:06:33.213727 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.213232 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7678ee9-03ce-4b69-81f1-8101576a1350-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "e7678ee9-03ce-4b69-81f1-8101576a1350" (UID: "e7678ee9-03ce-4b69-81f1-8101576a1350"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:06:33.213727 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.213379 2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e7678ee9-03ce-4b69-81f1-8101576a1350-alertmanager-main-db\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.213727 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.213402 2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7678ee9-03ce-4b69-81f1-8101576a1350-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.216359 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.216096 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7678ee9-03ce-4b69-81f1-8101576a1350-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e7678ee9-03ce-4b69-81f1-8101576a1350" (UID: "e7678ee9-03ce-4b69-81f1-8101576a1350"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:06:33.216359 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.216201 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "e7678ee9-03ce-4b69-81f1-8101576a1350" (UID: "e7678ee9-03ce-4b69-81f1-8101576a1350"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:33.216359 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.216213 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7678ee9-03ce-4b69-81f1-8101576a1350-config-out" (OuterVolumeSpecName: "config-out") pod "e7678ee9-03ce-4b69-81f1-8101576a1350" (UID: "e7678ee9-03ce-4b69-81f1-8101576a1350"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:06:33.216599 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.216467 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "e7678ee9-03ce-4b69-81f1-8101576a1350" (UID: "e7678ee9-03ce-4b69-81f1-8101576a1350"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:33.216599 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.216534 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7678ee9-03ce-4b69-81f1-8101576a1350-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "e7678ee9-03ce-4b69-81f1-8101576a1350" (UID: "e7678ee9-03ce-4b69-81f1-8101576a1350"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:06:33.216737 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.216592 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-config-volume" (OuterVolumeSpecName: "config-volume") pod "e7678ee9-03ce-4b69-81f1-8101576a1350" (UID: "e7678ee9-03ce-4b69-81f1-8101576a1350"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:33.216798 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.216758 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "e7678ee9-03ce-4b69-81f1-8101576a1350" (UID: "e7678ee9-03ce-4b69-81f1-8101576a1350"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:33.218392 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.218371 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7678ee9-03ce-4b69-81f1-8101576a1350-kube-api-access-z6js9" (OuterVolumeSpecName: "kube-api-access-z6js9") pod "e7678ee9-03ce-4b69-81f1-8101576a1350" (UID: "e7678ee9-03ce-4b69-81f1-8101576a1350"). InnerVolumeSpecName "kube-api-access-z6js9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:06:33.218633 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.218605 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "e7678ee9-03ce-4b69-81f1-8101576a1350" (UID: "e7678ee9-03ce-4b69-81f1-8101576a1350"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:33.220515 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.220488 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "e7678ee9-03ce-4b69-81f1-8101576a1350" (UID: "e7678ee9-03ce-4b69-81f1-8101576a1350"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:33.227872 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.227844 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-web-config" (OuterVolumeSpecName: "web-config") pod "e7678ee9-03ce-4b69-81f1-8101576a1350" (UID: "e7678ee9-03ce-4b69-81f1-8101576a1350"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:33.314429 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.314374 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.314429 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.314421 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-main-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.314429 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.314432 2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7678ee9-03ce-4b69-81f1-8101576a1350-config-out\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.314429 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.314441 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6js9\" (UniqueName: \"kubernetes.io/projected/e7678ee9-03ce-4b69-81f1-8101576a1350-kube-api-access-z6js9\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.314429 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.314450 2569 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7678ee9-03ce-4b69-81f1-8101576a1350-metrics-client-ca\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.314755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.314458 2569 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-cluster-tls-config\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.314755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.314466 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.314755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.314475 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.314755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.314484 2569 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-config-volume\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.314755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.314492 2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7678ee9-03ce-4b69-81f1-8101576a1350-web-config\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:33.314755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:33.314499 2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7678ee9-03ce-4b69-81f1-8101576a1350-tls-assets\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:34.028811 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.028777 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7678ee9-03ce-4b69-81f1-8101576a1350","Type":"ContainerDied","Data":"841503f5005226f10df4f9d9c6d31f9337baa750699b4e5547584cfac81a0c45"} Jul 09 18:06:34.029184 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.028833 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.029184 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.028879 2569 scope.go:117] "RemoveContainer" containerID="cef8076286ce61103eeba101e444821a22afdd9e011fac401d33266765209200" Jul 09 18:06:34.037167 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.037149 2569 scope.go:117] "RemoveContainer" containerID="06b6f532105bfebefe24a2fccec3e622628494825a40d0e4515f073817fb59cc" Jul 09 18:06:34.044255 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.044234 2569 scope.go:117] "RemoveContainer" containerID="45f0629b0ba4067f858be958cc6abe5d9d0edb5533141e9279bd0e2378fca0e1" Jul 09 18:06:34.051129 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.051015 2569 scope.go:117] "RemoveContainer" containerID="8af829a1248c53ccf8ae527ffbe500988203238fd274bcd67e0652a927e780f8" Jul 09 18:06:34.053183 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.053154 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:06:34.057000 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.056974 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:06:34.059606 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.059583 2569 scope.go:117] "RemoveContainer" containerID="a6718bf11c30b7a9fa159a35badc68b5b0cb324d152215d7c413212ca860d1d7" Jul 09 18:06:34.067763 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.067744 2569 scope.go:117] "RemoveContainer" containerID="2406ca5d4ac85dd8a3a10fc055e301f1357d2aba424d9ea645ff5e510c3517ae" Jul 09 18:06:34.078026 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.078015 2569 scope.go:117] "RemoveContainer" containerID="a304497ab8668f0c7f0d39592dab1a6a95fce6ebe3559b00fe3dd82277822b0c" Jul 09 18:06:34.085150 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085127 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:06:34.085483 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085469 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="prom-label-proxy" Jul 09 18:06:34.085541 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085486 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="prom-label-proxy" Jul 09 18:06:34.085541 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085500 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="kube-rbac-proxy" Jul 09 18:06:34.085541 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085505 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="kube-rbac-proxy" Jul 09 18:06:34.085541 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085517 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="init-config-reloader" Jul 09 18:06:34.085541 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085522 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="init-config-reloader" Jul 09 18:06:34.085541 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085532 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="alertmanager" Jul 09 18:06:34.085541 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085537 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="alertmanager" Jul 09 18:06:34.085541 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085544 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="kube-rbac-proxy-web" Jul 09 18:06:34.085907 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085549 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="kube-rbac-proxy-web" Jul 09 18:06:34.085907 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085559 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="kube-rbac-proxy-metric" Jul 09 18:06:34.085907 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085564 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="kube-rbac-proxy-metric" Jul 09 18:06:34.085907 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085573 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="config-reloader" Jul 09 18:06:34.085907 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085578 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="config-reloader" Jul 09 18:06:34.085907 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085631 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="alertmanager" Jul 09 18:06:34.085907 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085638 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="kube-rbac-proxy-metric" Jul 09 18:06:34.085907 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085646 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="kube-rbac-proxy" Jul 09 18:06:34.085907 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085652 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="prom-label-proxy" Jul 09 18:06:34.085907 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085660 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="kube-rbac-proxy-web" Jul 09 18:06:34.085907 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.085669 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" containerName="config-reloader" Jul 09 18:06:34.089144 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.089127 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.091937 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.091902 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 18:06:34.092072 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.091981 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 18:06:34.092072 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.092002 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 18:06:34.092072 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.091981 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 18:06:34.092072 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.092059 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 18:06:34.092308 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.092116 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 18:06:34.092456 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.092442 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 18:06:34.092529 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.092509 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rqd7s\"" Jul 09 18:06:34.092829 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.092813 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 18:06:34.098710 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.098675 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 18:06:34.104951 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.104929 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:06:34.196705 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.196609 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7678ee9-03ce-4b69-81f1-8101576a1350" path="/var/lib/kubelet/pods/e7678ee9-03ce-4b69-81f1-8101576a1350/volumes" Jul 09 18:06:34.220508 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.220476 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58a497b-9688-427a-b3dd-72c8614bcb2f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.220636 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.220515 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.220636 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.220538 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d58a497b-9688-427a-b3dd-72c8614bcb2f-config-out\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.220636 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.220595 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-config-volume\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.220836 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.220636 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.220836 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.220667 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.220836 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.220716 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-web-config\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.220836 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.220750 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d58a497b-9688-427a-b3dd-72c8614bcb2f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.220836 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.220773 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.220836 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.220821 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f9vz\" (UniqueName: \"kubernetes.io/projected/d58a497b-9688-427a-b3dd-72c8614bcb2f-kube-api-access-2f9vz\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.221024 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.220845 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d58a497b-9688-427a-b3dd-72c8614bcb2f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.221024 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.220861 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.221024 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.220877 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d58a497b-9688-427a-b3dd-72c8614bcb2f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.321909 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.321872 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58a497b-9688-427a-b3dd-72c8614bcb2f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.321909 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.321908 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.322137 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.321932 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d58a497b-9688-427a-b3dd-72c8614bcb2f-config-out\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.322137 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.321953 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-config-volume\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.322137 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.321998 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.322137 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.322043 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.322137 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.322081 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-web-config\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.322137 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.322114 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d58a497b-9688-427a-b3dd-72c8614bcb2f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.322385 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.322148 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.322385 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.322220 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f9vz\" (UniqueName: \"kubernetes.io/projected/d58a497b-9688-427a-b3dd-72c8614bcb2f-kube-api-access-2f9vz\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.322385 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.322258 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d58a497b-9688-427a-b3dd-72c8614bcb2f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.322385 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.322287 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.322385 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.322318 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d58a497b-9688-427a-b3dd-72c8614bcb2f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.323162 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.323137 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58a497b-9688-427a-b3dd-72c8614bcb2f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.323345 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.323199 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d58a497b-9688-427a-b3dd-72c8614bcb2f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.323478 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.323452 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d58a497b-9688-427a-b3dd-72c8614bcb2f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.325133 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.325109 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.325241 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.325205 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-config-volume\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.325951 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.325919 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.326017 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.325966 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-web-config\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.326017 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.325977 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d58a497b-9688-427a-b3dd-72c8614bcb2f-config-out\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.326175 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.326159 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.326445 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.326421 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d58a497b-9688-427a-b3dd-72c8614bcb2f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.326559 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.326542 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.327498 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.327481 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d58a497b-9688-427a-b3dd-72c8614bcb2f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.332242 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.332226 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f9vz\" (UniqueName: \"kubernetes.io/projected/d58a497b-9688-427a-b3dd-72c8614bcb2f-kube-api-access-2f9vz\") pod \"alertmanager-main-0\" (UID: \"d58a497b-9688-427a-b3dd-72c8614bcb2f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.399676 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.399648 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 18:06:34.548170 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:34.548146 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 18:06:34.550589 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:06:34.550564 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58a497b_9688_427a_b3dd_72c8614bcb2f.slice/crio-12bb8a1a087edeb79fe6ff48ff7fe207ba56c9d040a061aa7a7efab6f19a04bf WatchSource:0}: Error finding container 12bb8a1a087edeb79fe6ff48ff7fe207ba56c9d040a061aa7a7efab6f19a04bf: Status 404 returned error can't find the container with id 12bb8a1a087edeb79fe6ff48ff7fe207ba56c9d040a061aa7a7efab6f19a04bf Jul 09 18:06:35.034933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:35.034895 2569 generic.go:356] "Generic (PLEG): container finished" podID="d58a497b-9688-427a-b3dd-72c8614bcb2f" containerID="85094bf205dd64419f2a7d4df15a7e50fc647e52dd1b389866f33e9fb44329c2" exitCode=0 Jul 09 18:06:35.035331 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:35.034984 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58a497b-9688-427a-b3dd-72c8614bcb2f","Type":"ContainerDied","Data":"85094bf205dd64419f2a7d4df15a7e50fc647e52dd1b389866f33e9fb44329c2"} Jul 09 18:06:35.035331 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:35.035076 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58a497b-9688-427a-b3dd-72c8614bcb2f","Type":"ContainerStarted","Data":"12bb8a1a087edeb79fe6ff48ff7fe207ba56c9d040a061aa7a7efab6f19a04bf"} Jul 09 18:06:36.043699 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.043660 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58a497b-9688-427a-b3dd-72c8614bcb2f","Type":"ContainerStarted","Data":"859601665c1ba312333c33c82b0a4ea1e0f186211e54dda2f2c5c2a900c80ed1"} Jul 09 18:06:36.044260 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.043761 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58a497b-9688-427a-b3dd-72c8614bcb2f","Type":"ContainerStarted","Data":"79175c6d701d0490c8dda19954d99761690a4411ac3d20868ec63b454d68ef78"} Jul 09 18:06:36.044260 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.043803 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58a497b-9688-427a-b3dd-72c8614bcb2f","Type":"ContainerStarted","Data":"805e5a2dfde70e9bfa70697cb4cdad038f499ad3139eef46edb02671c4cf6c32"} Jul 09 18:06:36.044260 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.043849 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58a497b-9688-427a-b3dd-72c8614bcb2f","Type":"ContainerStarted","Data":"74ae8fcf9d96a13475d2bf663a0e02e39e0a727d627e75beed01266e36e54a45"} Jul 09 18:06:36.044260 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.043888 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58a497b-9688-427a-b3dd-72c8614bcb2f","Type":"ContainerStarted","Data":"a2ee7f194c0b6bd5e814f039fed0c733d69fc8ff599caba1e84170887a3236f4"} Jul 09 18:06:36.044260 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.043929 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58a497b-9688-427a-b3dd-72c8614bcb2f","Type":"ContainerStarted","Data":"8baafbfde67b885b39fb9bc98feaf94215e00c7c3f949de9768fe2239b1ab966"} Jul 09 18:06:36.077036 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.076963 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.060380528 podStartE2EDuration="2.076939529s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.559001ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:06:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:06:36.074548101 +0000 UTC m=+152.540070573" watchObservedRunningTime="2026-07-09 18:06:36.076939529 +0000 UTC m=+152.542461941" Jul 09 18:06:36.094799 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.094769 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:06:36.095416 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.095385 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="prometheus" containerID="cri-o://932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb" gracePeriod=600 Jul 09 18:06:36.095703 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.095432 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="kube-rbac-proxy-thanos" containerID="cri-o://a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549" gracePeriod=600 Jul 09 18:06:36.096801 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.095640 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="kube-rbac-proxy" containerID="cri-o://e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6" gracePeriod=600 Jul 09 18:06:36.097729 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.095714 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="kube-rbac-proxy-web" containerID="cri-o://affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630" gracePeriod=600 Jul 09 18:06:36.097729 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.095736 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="thanos-sidecar" containerID="cri-o://520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c" gracePeriod=600 Jul 09 18:06:36.097729 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.095756 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="config-reloader" containerID="cri-o://978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff" gracePeriod=600 Jul 09 18:06:36.377319 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.377013 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:36.543204 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543167 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-trusted-ca-bundle\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.543397 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543224 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.543397 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543248 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-2\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.543397 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543350 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqcxm\" (UniqueName: \"kubernetes.io/projected/39e0cb81-24ee-4743-903e-3d1114e0c8e7-kube-api-access-cqcxm\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.543397 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543389 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-kube-rbac-proxy\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.543616 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543419 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-grpc-tls\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.543616 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543454 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-db\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.543616 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543487 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-thanos-prometheus-http-client-file\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.543616 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543524 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-serving-certs-ca-bundle\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.543616 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543546 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-tls\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.543616 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543571 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39e0cb81-24ee-4743-903e-3d1114e0c8e7-config-out\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.543616 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543594 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-1\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.543616 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543595 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:06:36.544020 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543620 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-metrics-client-certs\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.544020 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543615 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:06:36.544020 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.543992 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:06:36.544172 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.544042 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.544172 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.544073 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-metrics-client-ca\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.544172 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.544130 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-0\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.544317 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.544167 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-config\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.544317 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.544210 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-kubelet-serving-ca-bundle\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.545087 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.544474 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:06:36.545087 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.544503 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-web-config\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.545087 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.544514 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:06:36.545087 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.544532 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39e0cb81-24ee-4743-903e-3d1114e0c8e7-tls-assets\") pod \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\" (UID: \"39e0cb81-24ee-4743-903e-3d1114e0c8e7\") " Jul 09 18:06:36.545087 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.544752 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:06:36.545388 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.545166 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-trusted-ca-bundle\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.545388 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.545186 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.545388 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.545201 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-db\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.545388 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.545216 2569 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.545388 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.545230 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.545388 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.545242 2569 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-metrics-client-ca\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.545669 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.545449 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:06:36.546401 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.546365 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:06:36.546657 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.546522 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39e0cb81-24ee-4743-903e-3d1114e0c8e7-kube-api-access-cqcxm" (OuterVolumeSpecName: "kube-api-access-cqcxm") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "kube-api-access-cqcxm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:06:36.546657 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.546592 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:36.546657 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.546628 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:36.547152 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.547114 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:36.547376 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.547349 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:36.548244 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.548218 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39e0cb81-24ee-4743-903e-3d1114e0c8e7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:06:36.548472 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.548447 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:36.548545 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.548472 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:36.548670 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.548647 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-config" (OuterVolumeSpecName: "config") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:36.548772 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.548730 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39e0cb81-24ee-4743-903e-3d1114e0c8e7-config-out" (OuterVolumeSpecName: "config-out") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:06:36.549474 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.549453 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:36.558892 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.558873 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-web-config" (OuterVolumeSpecName: "web-config") pod "39e0cb81-24ee-4743-903e-3d1114e0c8e7" (UID: "39e0cb81-24ee-4743-903e-3d1114e0c8e7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:06:36.646492 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.646414 2569 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-kube-rbac-proxy\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.646492 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.646449 2569 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-grpc-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.646492 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.646459 2569 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-thanos-prometheus-http-client-file\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.646492 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.646470 2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.646492 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.646478 2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39e0cb81-24ee-4743-903e-3d1114e0c8e7-config-out\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.646492 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.646487 2569 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-metrics-client-certs\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.646492 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.646496 2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.646492 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.646505 2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.646837 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.646514 2569 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-config\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.646837 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.646522 2569 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e0cb81-24ee-4743-903e-3d1114e0c8e7-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.646837 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.646530 2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-web-config\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.646837 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.646538 2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39e0cb81-24ee-4743-903e-3d1114e0c8e7-tls-assets\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.646837 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.646547 2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/39e0cb81-24ee-4743-903e-3d1114e0c8e7-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:36.646837 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:36.646555 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqcxm\" (UniqueName: \"kubernetes.io/projected/39e0cb81-24ee-4743-903e-3d1114e0c8e7-kube-api-access-cqcxm\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:06:37.050589 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050553 2569 generic.go:356] "Generic (PLEG): container finished" podID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerID="a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549" exitCode=0 Jul 09 18:06:37.050589 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050581 2569 generic.go:356] "Generic (PLEG): container finished" podID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerID="e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6" exitCode=0 Jul 09 18:06:37.050589 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050588 2569 generic.go:356] "Generic (PLEG): container finished" podID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerID="affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630" exitCode=0 Jul 09 18:06:37.050589 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050594 2569 generic.go:356] "Generic (PLEG): container finished" podID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerID="520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c" exitCode=0 Jul 09 18:06:37.050589 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050600 2569 generic.go:356] "Generic (PLEG): container finished" podID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerID="978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff" exitCode=0 Jul 09 18:06:37.051118 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050605 2569 generic.go:356] "Generic (PLEG): container finished" podID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerID="932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb" exitCode=0 Jul 09 18:06:37.051118 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050640 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.051118 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050673 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerDied","Data":"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549"} Jul 09 18:06:37.051118 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050769 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerDied","Data":"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6"} Jul 09 18:06:37.051118 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050813 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerDied","Data":"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630"} Jul 09 18:06:37.051118 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050833 2569 scope.go:117] "RemoveContainer" containerID="a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549" Jul 09 18:06:37.051118 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050854 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerDied","Data":"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c"} Jul 09 18:06:37.051118 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050899 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerDied","Data":"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff"} Jul 09 18:06:37.051118 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050939 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerDied","Data":"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb"} Jul 09 18:06:37.051118 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.050981 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39e0cb81-24ee-4743-903e-3d1114e0c8e7","Type":"ContainerDied","Data":"9fcdf59752ce15bd219e2f09f329c0504376e273aab3081c01c8c40fa9e6c74e"} Jul 09 18:06:37.060269 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.060227 2569 scope.go:117] "RemoveContainer" containerID="e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6" Jul 09 18:06:37.068353 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.068331 2569 scope.go:117] "RemoveContainer" containerID="affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630" Jul 09 18:06:37.075173 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.075152 2569 scope.go:117] "RemoveContainer" containerID="520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c" Jul 09 18:06:37.079189 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.079170 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:06:37.083539 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.083509 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:06:37.083789 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.083770 2569 scope.go:117] "RemoveContainer" containerID="978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff" Jul 09 18:06:37.092745 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.092724 2569 scope.go:117] "RemoveContainer" containerID="932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb" Jul 09 18:06:37.100137 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.100002 2569 scope.go:117] "RemoveContainer" containerID="26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4" Jul 09 18:06:37.106734 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.106715 2569 scope.go:117] "RemoveContainer" containerID="a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549" Jul 09 18:06:37.107015 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:06:37.106986 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549\": container with ID starting with a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549 not found: ID does not exist" containerID="a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549" Jul 09 18:06:37.107066 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.107017 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549"} err="failed to get container status \"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549\": rpc error: code = NotFound desc = could not find container \"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549\": container with ID starting with a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549 not found: ID does not exist" Jul 09 18:06:37.107066 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.107053 2569 scope.go:117] "RemoveContainer" containerID="e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6" Jul 09 18:06:37.107306 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:06:37.107280 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6\": container with ID starting with e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6 not found: ID does not exist" containerID="e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6" Jul 09 18:06:37.107372 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.107311 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6"} err="failed to get container status \"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6\": rpc error: code = NotFound desc = could not find container \"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6\": container with ID starting with e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6 not found: ID does not exist" Jul 09 18:06:37.107372 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.107325 2569 scope.go:117] "RemoveContainer" containerID="affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630" Jul 09 18:06:37.107542 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:06:37.107526 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630\": container with ID starting with affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630 not found: ID does not exist" containerID="affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630" Jul 09 18:06:37.107581 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.107548 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630"} err="failed to get container status \"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630\": rpc error: code = NotFound desc = could not find container \"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630\": container with ID starting with affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630 not found: ID does not exist" Jul 09 18:06:37.107581 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.107563 2569 scope.go:117] "RemoveContainer" containerID="520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c" Jul 09 18:06:37.107819 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:06:37.107803 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c\": container with ID starting with 520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c not found: ID does not exist" containerID="520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c" Jul 09 18:06:37.107875 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.107831 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c"} err="failed to get container status \"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c\": rpc error: code = NotFound desc = could not find container \"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c\": container with ID starting with 520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c not found: ID does not exist" Jul 09 18:06:37.107875 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.107844 2569 scope.go:117] "RemoveContainer" containerID="978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff" Jul 09 18:06:37.108060 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:06:37.108046 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff\": container with ID starting with 978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff not found: ID does not exist" containerID="978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff" Jul 09 18:06:37.108096 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.108063 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff"} err="failed to get container status \"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff\": rpc error: code = NotFound desc = could not find container \"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff\": container with ID starting with 978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff not found: ID does not exist" Jul 09 18:06:37.108096 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.108075 2569 scope.go:117] "RemoveContainer" containerID="932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb" Jul 09 18:06:37.108289 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:06:37.108272 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb\": container with ID starting with 932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb not found: ID does not exist" containerID="932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb" Jul 09 18:06:37.108339 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.108294 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb"} err="failed to get container status \"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb\": rpc error: code = NotFound desc = could not find container \"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb\": container with ID starting with 932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb not found: ID does not exist" Jul 09 18:06:37.108339 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.108314 2569 scope.go:117] "RemoveContainer" containerID="26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4" Jul 09 18:06:37.108590 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:06:37.108543 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4\": container with ID starting with 26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4 not found: ID does not exist" containerID="26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4" Jul 09 18:06:37.108590 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.108570 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4"} err="failed to get container status \"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4\": rpc error: code = NotFound desc = could not find container \"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4\": container with ID starting with 26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4 not found: ID does not exist" Jul 09 18:06:37.108590 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.108590 2569 scope.go:117] "RemoveContainer" containerID="a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549" Jul 09 18:06:37.109277 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.109205 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549"} err="failed to get container status \"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549\": rpc error: code = NotFound desc = could not find container \"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549\": container with ID starting with a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549 not found: ID does not exist" Jul 09 18:06:37.109277 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.109234 2569 scope.go:117] "RemoveContainer" containerID="e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6" Jul 09 18:06:37.109594 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.109566 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6"} err="failed to get container status \"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6\": rpc error: code = NotFound desc = could not find container \"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6\": container with ID starting with e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6 not found: ID does not exist" Jul 09 18:06:37.109713 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.109597 2569 scope.go:117] "RemoveContainer" containerID="affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630" Jul 09 18:06:37.109878 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.109855 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630"} err="failed to get container status \"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630\": rpc error: code = NotFound desc = could not find container \"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630\": container with ID starting with affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630 not found: ID does not exist" Jul 09 18:06:37.109878 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.109873 2569 scope.go:117] "RemoveContainer" containerID="520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c" Jul 09 18:06:37.110083 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.110068 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c"} err="failed to get container status \"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c\": rpc error: code = NotFound desc = could not find container \"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c\": container with ID starting with 520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c not found: ID does not exist" Jul 09 18:06:37.110126 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.110084 2569 scope.go:117] "RemoveContainer" containerID="978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff" Jul 09 18:06:37.110338 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.110318 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff"} err="failed to get container status \"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff\": rpc error: code = NotFound desc = could not find container \"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff\": container with ID starting with 978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff not found: ID does not exist" Jul 09 18:06:37.110411 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.110339 2569 scope.go:117] "RemoveContainer" containerID="932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb" Jul 09 18:06:37.110544 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.110527 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb"} err="failed to get container status \"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb\": rpc error: code = NotFound desc = could not find container \"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb\": container with ID starting with 932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb not found: ID does not exist" Jul 09 18:06:37.110613 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.110546 2569 scope.go:117] "RemoveContainer" containerID="26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4" Jul 09 18:06:37.110882 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.110864 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4"} err="failed to get container status \"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4\": rpc error: code = NotFound desc = could not find container \"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4\": container with ID starting with 26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4 not found: ID does not exist" Jul 09 18:06:37.110938 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.110883 2569 scope.go:117] "RemoveContainer" containerID="a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549" Jul 09 18:06:37.111078 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111059 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549"} err="failed to get container status \"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549\": rpc error: code = NotFound desc = could not find container \"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549\": container with ID starting with a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549 not found: ID does not exist" Jul 09 18:06:37.111123 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111079 2569 scope.go:117] "RemoveContainer" containerID="e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6" Jul 09 18:06:37.111199 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111179 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:06:37.111311 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111287 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6"} err="failed to get container status \"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6\": rpc error: code = NotFound desc = could not find container \"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6\": container with ID starting with e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6 not found: ID does not exist" Jul 09 18:06:37.111348 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111312 2569 scope.go:117] "RemoveContainer" containerID="affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630" Jul 09 18:06:37.111539 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111523 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630"} err="failed to get container status \"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630\": rpc error: code = NotFound desc = could not find container \"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630\": container with ID starting with affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630 not found: ID does not exist" Jul 09 18:06:37.111539 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111538 2569 scope.go:117] "RemoveContainer" containerID="520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c" Jul 09 18:06:37.111620 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111563 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="kube-rbac-proxy-web" Jul 09 18:06:37.111620 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111576 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="kube-rbac-proxy-web" Jul 09 18:06:37.111620 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111584 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="kube-rbac-proxy-thanos" Jul 09 18:06:37.111620 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111590 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="kube-rbac-proxy-thanos" Jul 09 18:06:37.111620 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111597 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="config-reloader" Jul 09 18:06:37.111620 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111604 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="config-reloader" Jul 09 18:06:37.111620 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111614 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="thanos-sidecar" Jul 09 18:06:37.111620 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111621 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="thanos-sidecar" Jul 09 18:06:37.111966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111637 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="prometheus" Jul 09 18:06:37.111966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111645 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="prometheus" Jul 09 18:06:37.111966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111662 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="init-config-reloader" Jul 09 18:06:37.111966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111672 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="init-config-reloader" Jul 09 18:06:37.111966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111716 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="kube-rbac-proxy" Jul 09 18:06:37.111966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111725 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="kube-rbac-proxy" Jul 09 18:06:37.111966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111758 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c"} err="failed to get container status \"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c\": rpc error: code = NotFound desc = could not find container \"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c\": container with ID starting with 520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c not found: ID does not exist" Jul 09 18:06:37.111966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111798 2569 scope.go:117] "RemoveContainer" containerID="978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff" Jul 09 18:06:37.111966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111799 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="prometheus" Jul 09 18:06:37.111966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111922 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="kube-rbac-proxy-thanos" Jul 09 18:06:37.111966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111934 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="thanos-sidecar" Jul 09 18:06:37.111966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111940 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="config-reloader" Jul 09 18:06:37.111966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111947 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="kube-rbac-proxy-web" Jul 09 18:06:37.111966 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.111953 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" containerName="kube-rbac-proxy" Jul 09 18:06:37.112543 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.112132 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff"} err="failed to get container status \"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff\": rpc error: code = NotFound desc = could not find container \"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff\": container with ID starting with 978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff not found: ID does not exist" Jul 09 18:06:37.112543 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.112167 2569 scope.go:117] "RemoveContainer" containerID="932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb" Jul 09 18:06:37.112543 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.112441 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb"} err="failed to get container status \"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb\": rpc error: code = NotFound desc = could not find container \"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb\": container with ID starting with 932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb not found: ID does not exist" Jul 09 18:06:37.112543 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.112457 2569 scope.go:117] "RemoveContainer" containerID="26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4" Jul 09 18:06:37.112799 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.112671 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4"} err="failed to get container status \"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4\": rpc error: code = NotFound desc = could not find container \"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4\": container with ID starting with 26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4 not found: ID does not exist" Jul 09 18:06:37.112799 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.112721 2569 scope.go:117] "RemoveContainer" containerID="a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549" Jul 09 18:06:37.113025 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.113004 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549"} err="failed to get container status \"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549\": rpc error: code = NotFound desc = could not find container \"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549\": container with ID starting with a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549 not found: ID does not exist" Jul 09 18:06:37.113064 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.113029 2569 scope.go:117] "RemoveContainer" containerID="e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6" Jul 09 18:06:37.113231 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.113214 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6"} err="failed to get container status \"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6\": rpc error: code = NotFound desc = could not find container \"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6\": container with ID starting with e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6 not found: ID does not exist" Jul 09 18:06:37.113300 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.113233 2569 scope.go:117] "RemoveContainer" containerID="affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630" Jul 09 18:06:37.113475 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.113456 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630"} err="failed to get container status \"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630\": rpc error: code = NotFound desc = could not find container \"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630\": container with ID starting with affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630 not found: ID does not exist" Jul 09 18:06:37.113546 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.113476 2569 scope.go:117] "RemoveContainer" containerID="520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c" Jul 09 18:06:37.113754 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.113732 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c"} err="failed to get container status \"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c\": rpc error: code = NotFound desc = could not find container \"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c\": container with ID starting with 520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c not found: ID does not exist" Jul 09 18:06:37.113814 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.113757 2569 scope.go:117] "RemoveContainer" containerID="978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff" Jul 09 18:06:37.113997 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.113978 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff"} err="failed to get container status \"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff\": rpc error: code = NotFound desc = could not find container \"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff\": container with ID starting with 978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff not found: ID does not exist" Jul 09 18:06:37.113997 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.113996 2569 scope.go:117] "RemoveContainer" containerID="932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb" Jul 09 18:06:37.114261 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.114242 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb"} err="failed to get container status \"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb\": rpc error: code = NotFound desc = could not find container \"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb\": container with ID starting with 932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb not found: ID does not exist" Jul 09 18:06:37.114313 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.114263 2569 scope.go:117] "RemoveContainer" containerID="26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4" Jul 09 18:06:37.114523 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.114493 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4"} err="failed to get container status \"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4\": rpc error: code = NotFound desc = could not find container \"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4\": container with ID starting with 26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4 not found: ID does not exist" Jul 09 18:06:37.114523 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.114514 2569 scope.go:117] "RemoveContainer" containerID="a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549" Jul 09 18:06:37.114771 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.114755 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549"} err="failed to get container status \"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549\": rpc error: code = NotFound desc = could not find container \"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549\": container with ID starting with a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549 not found: ID does not exist" Jul 09 18:06:37.114855 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.114773 2569 scope.go:117] "RemoveContainer" containerID="e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6" Jul 09 18:06:37.115023 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.115005 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6"} err="failed to get container status \"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6\": rpc error: code = NotFound desc = could not find container \"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6\": container with ID starting with e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6 not found: ID does not exist" Jul 09 18:06:37.115071 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.115024 2569 scope.go:117] "RemoveContainer" containerID="affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630" Jul 09 18:06:37.115230 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.115214 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630"} err="failed to get container status \"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630\": rpc error: code = NotFound desc = could not find container \"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630\": container with ID starting with affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630 not found: ID does not exist" Jul 09 18:06:37.115279 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.115230 2569 scope.go:117] "RemoveContainer" containerID="520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c" Jul 09 18:06:37.115460 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.115439 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c"} err="failed to get container status \"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c\": rpc error: code = NotFound desc = could not find container \"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c\": container with ID starting with 520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c not found: ID does not exist" Jul 09 18:06:37.115510 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.115461 2569 scope.go:117] "RemoveContainer" containerID="978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff" Jul 09 18:06:37.115665 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.115649 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff"} err="failed to get container status \"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff\": rpc error: code = NotFound desc = could not find container \"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff\": container with ID starting with 978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff not found: ID does not exist" Jul 09 18:06:37.115727 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.115665 2569 scope.go:117] "RemoveContainer" containerID="932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb" Jul 09 18:06:37.115890 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.115874 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb"} err="failed to get container status \"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb\": rpc error: code = NotFound desc = could not find container \"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb\": container with ID starting with 932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb not found: ID does not exist" Jul 09 18:06:37.115933 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.115892 2569 scope.go:117] "RemoveContainer" containerID="26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4" Jul 09 18:06:37.116094 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.116078 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4"} err="failed to get container status \"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4\": rpc error: code = NotFound desc = could not find container \"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4\": container with ID starting with 26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4 not found: ID does not exist" Jul 09 18:06:37.116130 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.116094 2569 scope.go:117] "RemoveContainer" containerID="a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549" Jul 09 18:06:37.116289 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.116271 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549"} err="failed to get container status \"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549\": rpc error: code = NotFound desc = could not find container \"a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549\": container with ID starting with a797d56012634ac006b1a889cf4acb1f9738fa1e0259a553efcd9377e5ef6549 not found: ID does not exist" Jul 09 18:06:37.116329 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.116290 2569 scope.go:117] "RemoveContainer" containerID="e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6" Jul 09 18:06:37.116462 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.116445 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.116520 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.116501 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6"} err="failed to get container status \"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6\": rpc error: code = NotFound desc = could not find container \"e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6\": container with ID starting with e00565178036f1f844e3ce5aa280fbb9becbd40139ed86418e77f61fe3079ef6 not found: ID does not exist" Jul 09 18:06:37.116576 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.116522 2569 scope.go:117] "RemoveContainer" containerID="affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630" Jul 09 18:06:37.117017 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.116988 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630"} err="failed to get container status \"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630\": rpc error: code = NotFound desc = could not find container \"affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630\": container with ID starting with affb9953c5d9d8a53ad04db417151773fce6538afe1919fdeb2c923fb8502630 not found: ID does not exist" Jul 09 18:06:37.117017 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.117017 2569 scope.go:117] "RemoveContainer" containerID="520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c" Jul 09 18:06:37.117298 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.117279 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c"} err="failed to get container status \"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c\": rpc error: code = NotFound desc = could not find container \"520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c\": container with ID starting with 520282d0f3f1f52976ce17100bf75be1b6c33d1aab0bf04739d99d94114d047c not found: ID does not exist" Jul 09 18:06:37.117344 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.117299 2569 scope.go:117] "RemoveContainer" containerID="978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff" Jul 09 18:06:37.117566 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.117546 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff"} err="failed to get container status \"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff\": rpc error: code = NotFound desc = could not find container \"978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff\": container with ID starting with 978eaa04f9fe3008aa1eb71d8b1420f75ba2c9aabdd8c0393317e1b896e302ff not found: ID does not exist" Jul 09 18:06:37.117646 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.117581 2569 scope.go:117] "RemoveContainer" containerID="932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb" Jul 09 18:06:37.117814 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.117795 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb"} err="failed to get container status \"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb\": rpc error: code = NotFound desc = could not find container \"932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb\": container with ID starting with 932b61819b2dfafa2454a9a69f76761b5be356a73798c0704032e929189d5dfb not found: ID does not exist" Jul 09 18:06:37.117921 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.117815 2569 scope.go:117] "RemoveContainer" containerID="26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4" Jul 09 18:06:37.118133 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.118107 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4"} err="failed to get container status \"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4\": rpc error: code = NotFound desc = could not find container \"26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4\": container with ID starting with 26499f1ab5030373b53a25c65cb8eb44d83276ef1b77c08f39d94b9fd3f2b5b4 not found: ID does not exist" Jul 09 18:06:37.119788 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.119442 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-w7cqr\"" Jul 09 18:06:37.119788 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.119525 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 18:06:37.119788 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.119566 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 18:06:37.119788 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.119447 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 18:06:37.120639 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.120344 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 18:06:37.120639 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.120497 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 18:06:37.120639 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.120513 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5is2hrefkin27\"" Jul 09 18:06:37.121528 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.120808 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 18:06:37.121528 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.120956 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 18:06:37.121528 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.121138 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 18:06:37.121528 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.121220 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 18:06:37.121528 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.121238 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 18:06:37.121528 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.121244 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 18:06:37.121528 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.121317 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 18:06:37.122849 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.122833 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 18:06:37.127545 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.127525 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 18:06:37.133166 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.133148 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:06:37.252837 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.252750 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.252837 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.252796 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.252837 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.252836 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253047 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.252854 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253047 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.252875 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-config-out\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253047 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.252894 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253047 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.252960 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253047 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.252989 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253047 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.253022 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253255 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.253068 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253255 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.253085 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253255 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.253101 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253255 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.253143 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253255 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.253168 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253255 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.253182 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253255 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.253202 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-config\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253255 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.253217 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253530 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.253265 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253530 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.253296 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-web-config\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.253530 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.253336 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlpv5\" (UniqueName: \"kubernetes.io/projected/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-kube-api-access-dlpv5\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.354714 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.354653 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.354714 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.354720 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.354924 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.354738 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.354924 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.354757 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-config\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.354924 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.354775 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.354924 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.354790 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.354924 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.354804 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-web-config\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.355145 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.354949 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlpv5\" (UniqueName: \"kubernetes.io/projected/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-kube-api-access-dlpv5\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.355145 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.355007 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.355145 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.355059 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.355145 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.355091 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.355145 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.355118 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.355395 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.355151 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-config-out\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.355395 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.355174 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.355395 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.355226 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.355395 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.355252 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.355395 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.355283 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.355395 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.355368 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.355885 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.355400 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.355885 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.355427 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.355885 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.355812 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.356338 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.356312 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.356338 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.356316 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.357204 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.356839 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.358104 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.358076 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.358217 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.358101 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.358280 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.358222 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.358280 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.358266 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.359082 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.359051 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.359170 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.359100 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.359930 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.359890 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.360554 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.360170 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.360554 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.360516 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.360554 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.360530 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.360910 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.360887 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-web-config\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.361029 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.361007 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-config\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.361070 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.361034 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.361176 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.361157 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.361360 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.361342 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-config-out\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.363973 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.363955 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlpv5\" (UniqueName: \"kubernetes.io/projected/b304d2bb-d80b-4d6d-8886-1d5c1e709e3b-kube-api-access-dlpv5\") pod \"prometheus-k8s-0\" (UID: \"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.428242 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.428213 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:37.565980 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:37.565950 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 18:06:37.569333 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:06:37.569302 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb304d2bb_d80b_4d6d_8886_1d5c1e709e3b.slice/crio-bc9589bf74f3d99795e1ce1944292e37727e0f3b2d8bf809ba5a0381666c3e57 WatchSource:0}: Error finding container bc9589bf74f3d99795e1ce1944292e37727e0f3b2d8bf809ba5a0381666c3e57: Status 404 returned error can't find the container with id bc9589bf74f3d99795e1ce1944292e37727e0f3b2d8bf809ba5a0381666c3e57 Jul 09 18:06:38.057254 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:38.057221 2569 generic.go:356] "Generic (PLEG): container finished" podID="b304d2bb-d80b-4d6d-8886-1d5c1e709e3b" containerID="86ee432b885ee4402c3c769a8249d3999fb13b944cb3a3d19208bb5e0fe886ac" exitCode=0 Jul 09 18:06:38.057713 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:38.057314 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b","Type":"ContainerDied","Data":"86ee432b885ee4402c3c769a8249d3999fb13b944cb3a3d19208bb5e0fe886ac"} Jul 09 18:06:38.057713 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:38.057381 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b","Type":"ContainerStarted","Data":"bc9589bf74f3d99795e1ce1944292e37727e0f3b2d8bf809ba5a0381666c3e57"} Jul 09 18:06:38.205664 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:38.205618 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39e0cb81-24ee-4743-903e-3d1114e0c8e7" path="/var/lib/kubelet/pods/39e0cb81-24ee-4743-903e-3d1114e0c8e7/volumes" Jul 09 18:06:39.065463 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:39.065430 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b","Type":"ContainerStarted","Data":"14945e38b656446d9df438cbe94b6c4125306490593b680bc0ad15c396b96b42"} Jul 09 18:06:39.065886 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:39.065492 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b","Type":"ContainerStarted","Data":"8385af6974162a7aadeb013ca1523b53056bdde16ac8730588e211fd6efd33cf"} Jul 09 18:06:39.065886 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:39.065518 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b","Type":"ContainerStarted","Data":"011de0bd58478d91d41c686332ee48a19d9f08a2282bff7214994ac447218a19"} Jul 09 18:06:39.065886 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:39.065544 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b","Type":"ContainerStarted","Data":"5dd17c7b8d7497a31d0cea9323e1801a6cf96319051f16c784c8d4ae739d1ac7"} Jul 09 18:06:39.065886 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:39.065569 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b","Type":"ContainerStarted","Data":"62a97979fd9a0bd97b1b0b20d3d94168351d5014aac479c66bc355f7e4ea8c9a"} Jul 09 18:06:39.065886 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:39.065591 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b304d2bb-d80b-4d6d-8886-1d5c1e709e3b","Type":"ContainerStarted","Data":"9a7bbbf8f0805f5ef1ac9ec6e30d8ff311a78275a2ca93147c0b6bbf2a7bc80e"} Jul 09 18:06:39.100383 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:39.100309 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.081277462 podStartE2EDuration="2.100292262s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.0148ms" isStatefulPod=false podCreationTimestamp="2026-07-09 18:06:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:06:39.097567825 +0000 UTC m=+155.563090259" watchObservedRunningTime="2026-07-09 18:06:39.100292262 +0000 UTC m=+155.565814672" Jul 09 18:06:39.349652 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:39.349574 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:06:39.409729 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:39.409676 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:06:42.429294 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:42.429266 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:06:48.359210 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:48.359175 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:06:55.824261 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:55.824226 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:06:55.925753 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:06:55.925719 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:18.051406 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:18.051371 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:18.152330 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:18.152301 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:18.252826 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:18.252792 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:18.520468 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:18.520437 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:18.772761 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:18.772641 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:20.568125 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:20.568097 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:20.669377 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:20.669349 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:29.100474 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:29.100436 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:29.202547 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:29.202511 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:29.268018 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:29.267984 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:37.428754 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:37.428709 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:07:37.445117 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:37.445089 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:07:38.345652 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:38.345343 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 18:07:38.919336 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:38.919285 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:39.018662 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:39.018628 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:39.353237 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:39.353207 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:39.455326 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:39.455295 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:39.556140 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:39.556107 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:07:53.123485 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:07:53.123431 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:08:27.628649 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:08:27.628524 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:08:27.725662 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:08:27.725630 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:08:34.166096 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:08:34.166063 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:08:34.469275 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:08:34.469174 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:08:34.885116 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:08:34.885089 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:08:35.007945 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:08:35.007905 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:08:35.106916 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:08:35.106879 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:08:35.434946 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:08:35.434915 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:08:35.569873 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:08:35.569809 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:08:35.793577 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:08:35.793531 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:08:35.888640 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:08:35.888607 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:08:36.081215 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:08:36.081125 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:08:36.259346 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:08:36.259302 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:08:36.361408 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:08:36.361338 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:09:03.997950 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:03.997925 2569 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 18:09:19.982197 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:19.982163 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:09:24.144267 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:24.144233 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:09:43.127478 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.127431 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:09:43.195555 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.195520 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-748bdf6488-j5trn"] Jul 09 18:09:43.198900 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.198883 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-748bdf6488-j5trn" Jul 09 18:09:43.201937 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.201895 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 09 18:09:43.202572 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.202552 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Jul 09 18:09:43.202641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.202568 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-dbg7w\"" Jul 09 18:09:43.202641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.202586 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 09 18:09:43.209496 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.209461 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:09:43.210241 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.210221 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-748bdf6488-j5trn"] Jul 09 18:09:43.305606 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.305568 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:09:43.382093 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.381993 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/551712af-c5c5-4422-bde5-492d209cb481-cert\") pod \"kserve-controller-manager-748bdf6488-j5trn\" (UID: \"551712af-c5c5-4422-bde5-492d209cb481\") " pod="kserve/kserve-controller-manager-748bdf6488-j5trn" Jul 09 18:09:43.382093 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.382047 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvj57\" (UniqueName: \"kubernetes.io/projected/551712af-c5c5-4422-bde5-492d209cb481-kube-api-access-lvj57\") pod \"kserve-controller-manager-748bdf6488-j5trn\" (UID: \"551712af-c5c5-4422-bde5-492d209cb481\") " pod="kserve/kserve-controller-manager-748bdf6488-j5trn" Jul 09 18:09:43.440613 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.440583 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:09:43.482554 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.482517 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvj57\" (UniqueName: \"kubernetes.io/projected/551712af-c5c5-4422-bde5-492d209cb481-kube-api-access-lvj57\") pod \"kserve-controller-manager-748bdf6488-j5trn\" (UID: \"551712af-c5c5-4422-bde5-492d209cb481\") " pod="kserve/kserve-controller-manager-748bdf6488-j5trn" Jul 09 18:09:43.482727 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.482653 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/551712af-c5c5-4422-bde5-492d209cb481-cert\") pod \"kserve-controller-manager-748bdf6488-j5trn\" (UID: \"551712af-c5c5-4422-bde5-492d209cb481\") " pod="kserve/kserve-controller-manager-748bdf6488-j5trn" Jul 09 18:09:43.482820 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:09:43.482800 2569 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 09 18:09:43.482886 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:09:43.482877 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/551712af-c5c5-4422-bde5-492d209cb481-cert podName:551712af-c5c5-4422-bde5-492d209cb481 nodeName:}" failed. No retries permitted until 2026-07-09 18:09:43.982859446 +0000 UTC m=+340.448381834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/551712af-c5c5-4422-bde5-492d209cb481-cert") pod "kserve-controller-manager-748bdf6488-j5trn" (UID: "551712af-c5c5-4422-bde5-492d209cb481") : secret "kserve-webhook-server-cert" not found Jul 09 18:09:43.492536 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.492511 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvj57\" (UniqueName: \"kubernetes.io/projected/551712af-c5c5-4422-bde5-492d209cb481-kube-api-access-lvj57\") pod \"kserve-controller-manager-748bdf6488-j5trn\" (UID: \"551712af-c5c5-4422-bde5-492d209cb481\") " pod="kserve/kserve-controller-manager-748bdf6488-j5trn" Jul 09 18:09:43.541259 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.541220 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:09:43.987500 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.987463 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/551712af-c5c5-4422-bde5-492d209cb481-cert\") pod \"kserve-controller-manager-748bdf6488-j5trn\" (UID: \"551712af-c5c5-4422-bde5-492d209cb481\") " pod="kserve/kserve-controller-manager-748bdf6488-j5trn" Jul 09 18:09:43.990153 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:43.990131 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/551712af-c5c5-4422-bde5-492d209cb481-cert\") pod \"kserve-controller-manager-748bdf6488-j5trn\" (UID: \"551712af-c5c5-4422-bde5-492d209cb481\") " pod="kserve/kserve-controller-manager-748bdf6488-j5trn" Jul 09 18:09:44.111007 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:44.110966 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-748bdf6488-j5trn" Jul 09 18:09:44.234023 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:44.233991 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:09:44.253516 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:44.253490 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-748bdf6488-j5trn"] Jul 09 18:09:44.256165 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:09:44.256128 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod551712af_c5c5_4422_bde5_492d209cb481.slice/crio-6fe1a30713fbb3ad875008dca898b8e62f56006239c9c06bf7f45e113f81dafa WatchSource:0}: Error finding container 6fe1a30713fbb3ad875008dca898b8e62f56006239c9c06bf7f45e113f81dafa: Status 404 returned error can't find the container with id 6fe1a30713fbb3ad875008dca898b8e62f56006239c9c06bf7f45e113f81dafa Jul 09 18:09:44.257753 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:44.257727 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 18:09:44.331704 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:44.331666 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:09:44.799308 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:44.799268 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-748bdf6488-j5trn" event={"ID":"551712af-c5c5-4422-bde5-492d209cb481","Type":"ContainerStarted","Data":"6fe1a30713fbb3ad875008dca898b8e62f56006239c9c06bf7f45e113f81dafa"} Jul 09 18:09:47.815437 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:47.815391 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-748bdf6488-j5trn" event={"ID":"551712af-c5c5-4422-bde5-492d209cb481","Type":"ContainerStarted","Data":"33ec322eeb60970e005dfb07d015e290d10fc0b6aa26edb36bc831ecac19c78f"} Jul 09 18:09:47.815857 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:47.815600 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-748bdf6488-j5trn" Jul 09 18:09:47.833158 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:47.833089 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-748bdf6488-j5trn" podStartSLOduration=1.785042107 podStartE2EDuration="4.833069608s" totalImagesPullingTime="3.048027501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:09:47.831571306 +0000 UTC m=+344.297093721" watchObservedRunningTime="2026-07-09 18:09:47.833069608 +0000 UTC m=+344.298592019" Jul 09 18:09:51.235801 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:51.235772 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:09:51.296879 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:09:51.296846 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:10:18.825771 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:18.825738 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-748bdf6488-j5trn" Jul 09 18:10:20.384795 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.384757 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-748bdf6488-j5trn"] Jul 09 18:10:20.385223 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.384970 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-748bdf6488-j5trn" podUID="551712af-c5c5-4422-bde5-492d209cb481" containerName="manager" containerID="cri-o://33ec322eeb60970e005dfb07d015e290d10fc0b6aa26edb36bc831ecac19c78f" gracePeriod=10 Jul 09 18:10:20.416228 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.416194 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-748bdf6488-fk24b"] Jul 09 18:10:20.421054 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.421032 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-748bdf6488-fk24b" Jul 09 18:10:20.430778 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.430748 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-748bdf6488-fk24b"] Jul 09 18:10:20.515546 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.515516 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/128800f2-1ef6-42f5-9d53-1bc6bc35121d-cert\") pod \"kserve-controller-manager-748bdf6488-fk24b\" (UID: \"128800f2-1ef6-42f5-9d53-1bc6bc35121d\") " pod="kserve/kserve-controller-manager-748bdf6488-fk24b" Jul 09 18:10:20.515713 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.515671 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc484\" (UniqueName: \"kubernetes.io/projected/128800f2-1ef6-42f5-9d53-1bc6bc35121d-kube-api-access-nc484\") pod \"kserve-controller-manager-748bdf6488-fk24b\" (UID: \"128800f2-1ef6-42f5-9d53-1bc6bc35121d\") " pod="kserve/kserve-controller-manager-748bdf6488-fk24b" Jul 09 18:10:20.617054 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.616936 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc484\" (UniqueName: \"kubernetes.io/projected/128800f2-1ef6-42f5-9d53-1bc6bc35121d-kube-api-access-nc484\") pod \"kserve-controller-manager-748bdf6488-fk24b\" (UID: \"128800f2-1ef6-42f5-9d53-1bc6bc35121d\") " pod="kserve/kserve-controller-manager-748bdf6488-fk24b" Jul 09 18:10:20.617054 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.616995 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/128800f2-1ef6-42f5-9d53-1bc6bc35121d-cert\") pod \"kserve-controller-manager-748bdf6488-fk24b\" (UID: \"128800f2-1ef6-42f5-9d53-1bc6bc35121d\") " pod="kserve/kserve-controller-manager-748bdf6488-fk24b" Jul 09 18:10:20.619583 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.619554 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/128800f2-1ef6-42f5-9d53-1bc6bc35121d-cert\") pod \"kserve-controller-manager-748bdf6488-fk24b\" (UID: \"128800f2-1ef6-42f5-9d53-1bc6bc35121d\") " pod="kserve/kserve-controller-manager-748bdf6488-fk24b" Jul 09 18:10:20.625566 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.625529 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc484\" (UniqueName: \"kubernetes.io/projected/128800f2-1ef6-42f5-9d53-1bc6bc35121d-kube-api-access-nc484\") pod \"kserve-controller-manager-748bdf6488-fk24b\" (UID: \"128800f2-1ef6-42f5-9d53-1bc6bc35121d\") " pod="kserve/kserve-controller-manager-748bdf6488-fk24b" Jul 09 18:10:20.656085 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.656010 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-748bdf6488-j5trn" Jul 09 18:10:20.779349 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.779319 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-748bdf6488-fk24b" Jul 09 18:10:20.818614 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.818581 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/551712af-c5c5-4422-bde5-492d209cb481-cert\") pod \"551712af-c5c5-4422-bde5-492d209cb481\" (UID: \"551712af-c5c5-4422-bde5-492d209cb481\") " Jul 09 18:10:20.818788 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.818682 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvj57\" (UniqueName: \"kubernetes.io/projected/551712af-c5c5-4422-bde5-492d209cb481-kube-api-access-lvj57\") pod \"551712af-c5c5-4422-bde5-492d209cb481\" (UID: \"551712af-c5c5-4422-bde5-492d209cb481\") " Jul 09 18:10:20.821031 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.821001 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/551712af-c5c5-4422-bde5-492d209cb481-cert" (OuterVolumeSpecName: "cert") pod "551712af-c5c5-4422-bde5-492d209cb481" (UID: "551712af-c5c5-4422-bde5-492d209cb481"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:10:20.821155 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.821023 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/551712af-c5c5-4422-bde5-492d209cb481-kube-api-access-lvj57" (OuterVolumeSpecName: "kube-api-access-lvj57") pod "551712af-c5c5-4422-bde5-492d209cb481" (UID: "551712af-c5c5-4422-bde5-492d209cb481"). InnerVolumeSpecName "kube-api-access-lvj57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:10:20.920424 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.920339 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvj57\" (UniqueName: \"kubernetes.io/projected/551712af-c5c5-4422-bde5-492d209cb481-kube-api-access-lvj57\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:10:20.920424 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.920369 2569 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/551712af-c5c5-4422-bde5-492d209cb481-cert\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:10:20.926831 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.926808 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-748bdf6488-fk24b"] Jul 09 18:10:20.929557 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:10:20.929530 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod128800f2_1ef6_42f5_9d53_1bc6bc35121d.slice/crio-491d23a5e327e4a96ea00dc9563795e76a1192fb03cd515a25406c49be718dce WatchSource:0}: Error finding container 491d23a5e327e4a96ea00dc9563795e76a1192fb03cd515a25406c49be718dce: Status 404 returned error can't find the container with id 491d23a5e327e4a96ea00dc9563795e76a1192fb03cd515a25406c49be718dce Jul 09 18:10:20.948756 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.948723 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-748bdf6488-fk24b" event={"ID":"128800f2-1ef6-42f5-9d53-1bc6bc35121d","Type":"ContainerStarted","Data":"491d23a5e327e4a96ea00dc9563795e76a1192fb03cd515a25406c49be718dce"} Jul 09 18:10:20.949851 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.949821 2569 generic.go:356] "Generic (PLEG): container finished" podID="551712af-c5c5-4422-bde5-492d209cb481" containerID="33ec322eeb60970e005dfb07d015e290d10fc0b6aa26edb36bc831ecac19c78f" exitCode=0 Jul 09 18:10:20.949981 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.949871 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-748bdf6488-j5trn" Jul 09 18:10:20.949981 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.949910 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-748bdf6488-j5trn" event={"ID":"551712af-c5c5-4422-bde5-492d209cb481","Type":"ContainerDied","Data":"33ec322eeb60970e005dfb07d015e290d10fc0b6aa26edb36bc831ecac19c78f"} Jul 09 18:10:20.949981 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.949966 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-748bdf6488-j5trn" event={"ID":"551712af-c5c5-4422-bde5-492d209cb481","Type":"ContainerDied","Data":"6fe1a30713fbb3ad875008dca898b8e62f56006239c9c06bf7f45e113f81dafa"} Jul 09 18:10:20.950148 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.949996 2569 scope.go:117] "RemoveContainer" containerID="33ec322eeb60970e005dfb07d015e290d10fc0b6aa26edb36bc831ecac19c78f" Jul 09 18:10:20.958704 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.958671 2569 scope.go:117] "RemoveContainer" containerID="33ec322eeb60970e005dfb07d015e290d10fc0b6aa26edb36bc831ecac19c78f" Jul 09 18:10:20.958979 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:10:20.958961 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33ec322eeb60970e005dfb07d015e290d10fc0b6aa26edb36bc831ecac19c78f\": container with ID starting with 33ec322eeb60970e005dfb07d015e290d10fc0b6aa26edb36bc831ecac19c78f not found: ID does not exist" containerID="33ec322eeb60970e005dfb07d015e290d10fc0b6aa26edb36bc831ecac19c78f" Jul 09 18:10:20.959031 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.958987 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33ec322eeb60970e005dfb07d015e290d10fc0b6aa26edb36bc831ecac19c78f"} err="failed to get container status \"33ec322eeb60970e005dfb07d015e290d10fc0b6aa26edb36bc831ecac19c78f\": rpc error: code = NotFound desc = could not find container \"33ec322eeb60970e005dfb07d015e290d10fc0b6aa26edb36bc831ecac19c78f\": container with ID starting with 33ec322eeb60970e005dfb07d015e290d10fc0b6aa26edb36bc831ecac19c78f not found: ID does not exist" Jul 09 18:10:20.971670 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.971637 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-748bdf6488-j5trn"] Jul 09 18:10:20.981032 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:20.980016 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-748bdf6488-j5trn"] Jul 09 18:10:21.955033 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:21.954999 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-748bdf6488-fk24b" event={"ID":"128800f2-1ef6-42f5-9d53-1bc6bc35121d","Type":"ContainerStarted","Data":"e58b43061170a6b4d7f3bf618c12077799dac0d4d309ce40612c8b583e4e3123"} Jul 09 18:10:21.955514 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:21.955082 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-748bdf6488-fk24b" Jul 09 18:10:21.972964 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:21.972913 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-748bdf6488-fk24b" podStartSLOduration=1.378914144 podStartE2EDuration="1.972895077s" totalImagesPullingTime="593.980933ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:10:21.970999263 +0000 UTC m=+378.436521677" watchObservedRunningTime="2026-07-09 18:10:21.972895077 +0000 UTC m=+378.438417487" Jul 09 18:10:22.196791 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:22.196762 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="551712af-c5c5-4422-bde5-492d209cb481" path="/var/lib/kubelet/pods/551712af-c5c5-4422-bde5-492d209cb481/volumes" Jul 09 18:10:43.508449 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:43.508410 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:10:43.586144 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:43.586106 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:10:52.965858 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:52.965822 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-748bdf6488-fk24b" Jul 09 18:10:54.064761 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.064641 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:10:54.134241 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.134200 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-lsdt4"] Jul 09 18:10:54.134666 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.134646 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="551712af-c5c5-4422-bde5-492d209cb481" containerName="manager" Jul 09 18:10:54.134799 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.134669 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="551712af-c5c5-4422-bde5-492d209cb481" containerName="manager" Jul 09 18:10:54.134799 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.134784 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="551712af-c5c5-4422-bde5-492d209cb481" containerName="manager" Jul 09 18:10:54.137106 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.137079 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-lsdt4" Jul 09 18:10:54.139612 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.139586 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-4x7jj\"" Jul 09 18:10:54.139875 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.139859 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\"" Jul 09 18:10:54.148856 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.148824 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-lsdt4"] Jul 09 18:10:54.151140 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.151108 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:10:54.218814 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.218778 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bae04c58-8e7f-4ea1-866e-15a6e8b51f9e-cert\") pod \"odh-model-controller-84b54b7f97-lsdt4\" (UID: \"bae04c58-8e7f-4ea1-866e-15a6e8b51f9e\") " pod="kserve/odh-model-controller-84b54b7f97-lsdt4" Jul 09 18:10:54.218814 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.218822 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gk2d\" (UniqueName: \"kubernetes.io/projected/bae04c58-8e7f-4ea1-866e-15a6e8b51f9e-kube-api-access-4gk2d\") pod \"odh-model-controller-84b54b7f97-lsdt4\" (UID: \"bae04c58-8e7f-4ea1-866e-15a6e8b51f9e\") " pod="kserve/odh-model-controller-84b54b7f97-lsdt4" Jul 09 18:10:54.235588 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.235558 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:10:54.319459 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.319357 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gk2d\" (UniqueName: \"kubernetes.io/projected/bae04c58-8e7f-4ea1-866e-15a6e8b51f9e-kube-api-access-4gk2d\") pod \"odh-model-controller-84b54b7f97-lsdt4\" (UID: \"bae04c58-8e7f-4ea1-866e-15a6e8b51f9e\") " pod="kserve/odh-model-controller-84b54b7f97-lsdt4" Jul 09 18:10:54.319666 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.319501 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bae04c58-8e7f-4ea1-866e-15a6e8b51f9e-cert\") pod \"odh-model-controller-84b54b7f97-lsdt4\" (UID: \"bae04c58-8e7f-4ea1-866e-15a6e8b51f9e\") " pod="kserve/odh-model-controller-84b54b7f97-lsdt4" Jul 09 18:10:54.322202 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.322168 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bae04c58-8e7f-4ea1-866e-15a6e8b51f9e-cert\") pod \"odh-model-controller-84b54b7f97-lsdt4\" (UID: \"bae04c58-8e7f-4ea1-866e-15a6e8b51f9e\") " pod="kserve/odh-model-controller-84b54b7f97-lsdt4" Jul 09 18:10:54.329088 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.329057 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gk2d\" (UniqueName: \"kubernetes.io/projected/bae04c58-8e7f-4ea1-866e-15a6e8b51f9e-kube-api-access-4gk2d\") pod \"odh-model-controller-84b54b7f97-lsdt4\" (UID: \"bae04c58-8e7f-4ea1-866e-15a6e8b51f9e\") " pod="kserve/odh-model-controller-84b54b7f97-lsdt4" Jul 09 18:10:54.335248 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.335217 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:10:54.451560 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.451520 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-lsdt4" Jul 09 18:10:54.599477 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.599280 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-lsdt4"] Jul 09 18:10:54.602494 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:10:54.602461 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbae04c58_8e7f_4ea1_866e_15a6e8b51f9e.slice/crio-6e9b5aac8d11845a24950c2ee6967f0554c6942009c50196c6be116ca15ecb44 WatchSource:0}: Error finding container 6e9b5aac8d11845a24950c2ee6967f0554c6942009c50196c6be116ca15ecb44: Status 404 returned error can't find the container with id 6e9b5aac8d11845a24950c2ee6967f0554c6942009c50196c6be116ca15ecb44 Jul 09 18:10:54.891657 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.891563 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:10:54.994870 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:54.994841 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:10:55.092808 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:55.092777 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:10:55.099042 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:55.099008 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-lsdt4" event={"ID":"bae04c58-8e7f-4ea1-866e-15a6e8b51f9e","Type":"ContainerStarted","Data":"6e9b5aac8d11845a24950c2ee6967f0554c6942009c50196c6be116ca15ecb44"} Jul 09 18:10:55.151738 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:55.151614 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:10:58.117680 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:58.117640 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-lsdt4" event={"ID":"bae04c58-8e7f-4ea1-866e-15a6e8b51f9e","Type":"ContainerStarted","Data":"7b05be272e2b7246d19e6043c8d76f73fbc040a32a814386c5764195893ec3ca"} Jul 09 18:10:58.118086 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:58.117762 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-lsdt4" Jul 09 18:10:58.138148 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:10:58.138081 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-lsdt4" podStartSLOduration=1.457707056 podStartE2EDuration="4.138062232s" totalImagesPullingTime="2.680355176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:10:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:10:58.136931559 +0000 UTC m=+414.602453978" watchObservedRunningTime="2026-07-09 18:10:58.138062232 +0000 UTC m=+414.603584643" Jul 09 18:11:09.125418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:09.125385 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-lsdt4" Jul 09 18:11:10.230433 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:10.230394 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-74tq2"] Jul 09 18:11:10.233484 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:10.233461 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-74tq2" Jul 09 18:11:10.237338 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:10.237311 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-tj2ht\"" Jul 09 18:11:10.237510 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:10.237357 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 09 18:11:10.251382 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:10.251348 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-74tq2"] Jul 09 18:11:10.367790 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:10.367738 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vkgh\" (UniqueName: \"kubernetes.io/projected/6244a66e-5dc5-4928-9140-4685cb2a7372-kube-api-access-2vkgh\") pod \"s3-init-74tq2\" (UID: \"6244a66e-5dc5-4928-9140-4685cb2a7372\") " pod="kserve/s3-init-74tq2" Jul 09 18:11:10.367982 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:10.367851 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6244a66e-5dc5-4928-9140-4685cb2a7372-model-cache\") pod \"s3-init-74tq2\" (UID: \"6244a66e-5dc5-4928-9140-4685cb2a7372\") " pod="kserve/s3-init-74tq2" Jul 09 18:11:10.469433 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:10.469391 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vkgh\" (UniqueName: \"kubernetes.io/projected/6244a66e-5dc5-4928-9140-4685cb2a7372-kube-api-access-2vkgh\") pod \"s3-init-74tq2\" (UID: \"6244a66e-5dc5-4928-9140-4685cb2a7372\") " pod="kserve/s3-init-74tq2" Jul 09 18:11:10.469614 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:10.469449 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6244a66e-5dc5-4928-9140-4685cb2a7372-model-cache\") pod \"s3-init-74tq2\" (UID: \"6244a66e-5dc5-4928-9140-4685cb2a7372\") " pod="kserve/s3-init-74tq2" Jul 09 18:11:10.469861 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:10.469846 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6244a66e-5dc5-4928-9140-4685cb2a7372-model-cache\") pod \"s3-init-74tq2\" (UID: \"6244a66e-5dc5-4928-9140-4685cb2a7372\") " pod="kserve/s3-init-74tq2" Jul 09 18:11:10.480777 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:10.480704 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vkgh\" (UniqueName: \"kubernetes.io/projected/6244a66e-5dc5-4928-9140-4685cb2a7372-kube-api-access-2vkgh\") pod \"s3-init-74tq2\" (UID: \"6244a66e-5dc5-4928-9140-4685cb2a7372\") " pod="kserve/s3-init-74tq2" Jul 09 18:11:10.559533 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:10.559495 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-74tq2" Jul 09 18:11:10.720018 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:10.719986 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-74tq2"] Jul 09 18:11:10.724071 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:11:10.724040 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6244a66e_5dc5_4928_9140_4685cb2a7372.slice/crio-10230e711bf2e749a5f86b25304056bc3642599bb9810b7b666b5b738dc96c96 WatchSource:0}: Error finding container 10230e711bf2e749a5f86b25304056bc3642599bb9810b7b666b5b738dc96c96: Status 404 returned error can't find the container with id 10230e711bf2e749a5f86b25304056bc3642599bb9810b7b666b5b738dc96c96 Jul 09 18:11:11.173418 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:11.173385 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-74tq2" event={"ID":"6244a66e-5dc5-4928-9140-4685cb2a7372","Type":"ContainerStarted","Data":"10230e711bf2e749a5f86b25304056bc3642599bb9810b7b666b5b738dc96c96"} Jul 09 18:11:17.201417 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:17.201371 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-74tq2" event={"ID":"6244a66e-5dc5-4928-9140-4685cb2a7372","Type":"ContainerStarted","Data":"c2cbc6b40fe1aa897d0cdeb1d6b8300a4a14fdaefa984d7888b0fcc3522a9d7b"} Jul 09 18:11:18.207431 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:18.207312 2569 generic.go:356] "Generic (PLEG): container finished" podID="6244a66e-5dc5-4928-9140-4685cb2a7372" containerID="c2cbc6b40fe1aa897d0cdeb1d6b8300a4a14fdaefa984d7888b0fcc3522a9d7b" exitCode=0 Jul 09 18:11:18.207431 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:18.207401 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-74tq2" event={"ID":"6244a66e-5dc5-4928-9140-4685cb2a7372","Type":"ContainerDied","Data":"c2cbc6b40fe1aa897d0cdeb1d6b8300a4a14fdaefa984d7888b0fcc3522a9d7b"} Jul 09 18:11:23.236231 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:23.236148 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-74tq2" event={"ID":"6244a66e-5dc5-4928-9140-4685cb2a7372","Type":"ContainerStarted","Data":"ce580e191975d90a89c72a3ba5486f131ce4ba9dd0ad79b44906d61175a54501"} Jul 09 18:11:23.259416 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:11:23.259355 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-74tq2" podStartSLOduration=1.563528567 podStartE2EDuration="13.259327518s" totalImagesPullingTime="10.134305508s" totalInitContainerRuntime="1.561493443s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:11:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:11:23.258996415 +0000 UTC m=+439.724518826" watchObservedRunningTime="2026-07-09 18:11:23.259327518 +0000 UTC m=+439.724849928" Jul 09 18:12:29.523954 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:29.523920 2569 generic.go:356] "Generic (PLEG): container finished" podID="6244a66e-5dc5-4928-9140-4685cb2a7372" containerID="ce580e191975d90a89c72a3ba5486f131ce4ba9dd0ad79b44906d61175a54501" exitCode=0 Jul 09 18:12:29.524367 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:29.523965 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-74tq2" event={"ID":"6244a66e-5dc5-4928-9140-4685cb2a7372","Type":"ContainerDied","Data":"ce580e191975d90a89c72a3ba5486f131ce4ba9dd0ad79b44906d61175a54501"} Jul 09 18:12:30.661298 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:30.661267 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-74tq2" Jul 09 18:12:30.814265 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:30.814178 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vkgh\" (UniqueName: \"kubernetes.io/projected/6244a66e-5dc5-4928-9140-4685cb2a7372-kube-api-access-2vkgh\") pod \"6244a66e-5dc5-4928-9140-4685cb2a7372\" (UID: \"6244a66e-5dc5-4928-9140-4685cb2a7372\") " Jul 09 18:12:30.814265 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:30.814225 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6244a66e-5dc5-4928-9140-4685cb2a7372-model-cache\") pod \"6244a66e-5dc5-4928-9140-4685cb2a7372\" (UID: \"6244a66e-5dc5-4928-9140-4685cb2a7372\") " Jul 09 18:12:30.816611 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:30.816574 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6244a66e-5dc5-4928-9140-4685cb2a7372-kube-api-access-2vkgh" (OuterVolumeSpecName: "kube-api-access-2vkgh") pod "6244a66e-5dc5-4928-9140-4685cb2a7372" (UID: "6244a66e-5dc5-4928-9140-4685cb2a7372"). InnerVolumeSpecName "kube-api-access-2vkgh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:12:30.833749 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:30.833681 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6244a66e-5dc5-4928-9140-4685cb2a7372-model-cache" (OuterVolumeSpecName: "model-cache") pod "6244a66e-5dc5-4928-9140-4685cb2a7372" (UID: "6244a66e-5dc5-4928-9140-4685cb2a7372"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:12:30.915456 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:30.915419 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vkgh\" (UniqueName: \"kubernetes.io/projected/6244a66e-5dc5-4928-9140-4685cb2a7372-kube-api-access-2vkgh\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:12:30.915456 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:30.915449 2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6244a66e-5dc5-4928-9140-4685cb2a7372-model-cache\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:12:31.534114 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:31.534078 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-74tq2" Jul 09 18:12:31.534296 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:31.534077 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-74tq2" event={"ID":"6244a66e-5dc5-4928-9140-4685cb2a7372","Type":"ContainerDied","Data":"10230e711bf2e749a5f86b25304056bc3642599bb9810b7b666b5b738dc96c96"} Jul 09 18:12:31.534296 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:31.534193 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10230e711bf2e749a5f86b25304056bc3642599bb9810b7b666b5b738dc96c96" Jul 09 18:12:32.031748 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:32.031705 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:12:32.110570 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:32.110538 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:12:44.155221 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:44.155174 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:12:44.256460 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:12:44.256425 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:13:30.681058 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:13:30.681023 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:14:39.479414 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.479333 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:14:39.648955 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.648923 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x"] Jul 09 18:14:39.649430 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.649409 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6244a66e-5dc5-4928-9140-4685cb2a7372" containerName="download-hf-model" Jul 09 18:14:39.649430 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.649428 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6244a66e-5dc5-4928-9140-4685cb2a7372" containerName="download-hf-model" Jul 09 18:14:39.649608 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.649440 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6244a66e-5dc5-4928-9140-4685cb2a7372" containerName="s3-init" Jul 09 18:14:39.649608 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.649446 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6244a66e-5dc5-4928-9140-4685cb2a7372" containerName="s3-init" Jul 09 18:14:39.649608 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.649512 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6244a66e-5dc5-4928-9140-4685cb2a7372" containerName="s3-init" Jul 09 18:14:39.651669 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.651652 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:39.654011 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.653989 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-a2b25-predictor-serving-cert\"" Jul 09 18:14:39.654112 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.654050 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 09 18:14:39.654267 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.654251 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-a2b25-kube-rbac-proxy-sar-config\"" Jul 09 18:14:39.654335 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.654317 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-8d4cq\"" Jul 09 18:14:39.654391 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.654337 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 09 18:14:39.654972 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.654958 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 09 18:14:39.655044 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.654985 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 09 18:14:39.662248 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.662229 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x"] Jul 09 18:14:39.750309 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.750233 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9805b5d0-6083-495b-aacf-7396ef8dc11d-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:39.750309 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.750293 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9805b5d0-6083-495b-aacf-7396ef8dc11d-proxy-tls\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:39.750484 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.750343 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9805b5d0-6083-495b-aacf-7396ef8dc11d-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:39.750484 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.750432 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9805b5d0-6083-495b-aacf-7396ef8dc11d-cabundle-cert\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:39.750484 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.750476 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9stl\" (UniqueName: \"kubernetes.io/projected/9805b5d0-6083-495b-aacf-7396ef8dc11d-kube-api-access-p9stl\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:39.851928 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.851888 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9805b5d0-6083-495b-aacf-7396ef8dc11d-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:39.852131 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.851948 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9805b5d0-6083-495b-aacf-7396ef8dc11d-proxy-tls\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:39.852131 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.851968 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9805b5d0-6083-495b-aacf-7396ef8dc11d-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:39.852131 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.852031 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9805b5d0-6083-495b-aacf-7396ef8dc11d-cabundle-cert\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:39.852131 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:14:39.852086 2569 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-serving-cert: secret "isvc-xgboost-graph-raw-a2b25-predictor-serving-cert" not found Jul 09 18:14:39.852345 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:14:39.852170 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9805b5d0-6083-495b-aacf-7396ef8dc11d-proxy-tls podName:9805b5d0-6083-495b-aacf-7396ef8dc11d nodeName:}" failed. No retries permitted until 2026-07-09 18:14:40.352144423 +0000 UTC m=+636.817666813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/9805b5d0-6083-495b-aacf-7396ef8dc11d-proxy-tls") pod "isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" (UID: "9805b5d0-6083-495b-aacf-7396ef8dc11d") : secret "isvc-xgboost-graph-raw-a2b25-predictor-serving-cert" not found Jul 09 18:14:39.852345 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.852089 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9stl\" (UniqueName: \"kubernetes.io/projected/9805b5d0-6083-495b-aacf-7396ef8dc11d-kube-api-access-p9stl\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:39.852345 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.852297 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9805b5d0-6083-495b-aacf-7396ef8dc11d-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:39.852779 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.852757 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9805b5d0-6083-495b-aacf-7396ef8dc11d-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:39.852902 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.852848 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9805b5d0-6083-495b-aacf-7396ef8dc11d-cabundle-cert\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:39.861322 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:39.861294 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9stl\" (UniqueName: \"kubernetes.io/projected/9805b5d0-6083-495b-aacf-7396ef8dc11d-kube-api-access-p9stl\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:40.357262 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:40.357222 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9805b5d0-6083-495b-aacf-7396ef8dc11d-proxy-tls\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:40.359869 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:40.359837 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9805b5d0-6083-495b-aacf-7396ef8dc11d-proxy-tls\") pod \"isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:40.563652 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:40.563614 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:14:40.703456 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:40.703430 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x"] Jul 09 18:14:40.706865 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:14:40.706837 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9805b5d0_6083_495b_aacf_7396ef8dc11d.slice/crio-fef0882ea7452b876dbfd478253b45c80ff56ea45b5cd02c22c70eca279ac680 WatchSource:0}: Error finding container fef0882ea7452b876dbfd478253b45c80ff56ea45b5cd02c22c70eca279ac680: Status 404 returned error can't find the container with id fef0882ea7452b876dbfd478253b45c80ff56ea45b5cd02c22c70eca279ac680 Jul 09 18:14:41.090235 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:41.090197 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" event={"ID":"9805b5d0-6083-495b-aacf-7396ef8dc11d","Type":"ContainerStarted","Data":"8ca3f5f06356664fc9ec57cffa5fe8120b3ad44c8677afba84a688a8fff1f6b2"} Jul 09 18:14:41.090509 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:41.090256 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" event={"ID":"9805b5d0-6083-495b-aacf-7396ef8dc11d","Type":"ContainerStarted","Data":"fef0882ea7452b876dbfd478253b45c80ff56ea45b5cd02c22c70eca279ac680"} Jul 09 18:14:41.106304 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:41.106275 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:14:45.109539 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:45.109508 2569 generic.go:356] "Generic (PLEG): container finished" podID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerID="8ca3f5f06356664fc9ec57cffa5fe8120b3ad44c8677afba84a688a8fff1f6b2" exitCode=0 Jul 09 18:14:45.109935 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:45.109580 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" event={"ID":"9805b5d0-6083-495b-aacf-7396ef8dc11d","Type":"ContainerDied","Data":"8ca3f5f06356664fc9ec57cffa5fe8120b3ad44c8677afba84a688a8fff1f6b2"} Jul 09 18:14:45.110895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:14:45.110880 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 18:15:00.082880 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:00.082844 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:15:00.180896 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:00.180853 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:15:01.474021 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:01.473964 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:15:06.216784 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:06.216747 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" event={"ID":"9805b5d0-6083-495b-aacf-7396ef8dc11d","Type":"ContainerStarted","Data":"d7367a0055de286c2750a2239753577f1bd840308b3786033f6ad72a0723d6d8"} Jul 09 18:15:09.233153 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:09.233112 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" event={"ID":"9805b5d0-6083-495b-aacf-7396ef8dc11d","Type":"ContainerStarted","Data":"6a03ee713d1e94c53d3781f6b67479f08fe4a84ea42b19e36671b087572cce80"} Jul 09 18:15:09.233635 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:09.233183 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:15:09.233635 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:09.233329 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:15:09.234461 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:09.234434 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Jul 09 18:15:09.256256 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:09.256213 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" podStartSLOduration=3.614968232 podStartE2EDuration="30.256199408s" totalImagesPullingTime="23.016852301s" totalInitContainerRuntime="3.624378875s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:14:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:15:09.253870064 +0000 UTC m=+665.719392505" watchObservedRunningTime="2026-07-09 18:15:09.256199408 +0000 UTC m=+665.721721819" Jul 09 18:15:10.237604 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:10.237564 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Jul 09 18:15:15.242498 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:15.242461 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:15:15.243098 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:15.243049 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Jul 09 18:15:25.242953 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:25.242912 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Jul 09 18:15:35.243915 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:35.243869 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Jul 09 18:15:45.243674 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:45.243631 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Jul 09 18:15:55.243238 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:15:55.243156 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Jul 09 18:16:05.244484 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:05.244454 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:16:19.658675 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:19.658636 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:16:20.367361 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:20.367316 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:16:24.878218 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:24.878179 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:16:29.678479 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:29.678448 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:16:29.862099 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:29.862066 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:16:29.950022 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:29.949938 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:16:29.988125 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:29.988093 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x"] Jul 09 18:16:29.988369 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:29.988332 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kserve-container" containerID="cri-o://d7367a0055de286c2750a2239753577f1bd840308b3786033f6ad72a0723d6d8" gracePeriod=30 Jul 09 18:16:29.988504 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:29.988376 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kube-rbac-proxy" containerID="cri-o://6a03ee713d1e94c53d3781f6b67479f08fe4a84ea42b19e36671b087572cce80" gracePeriod=30 Jul 09 18:16:30.011913 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.011889 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq"] Jul 09 18:16:30.015114 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.015092 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.017434 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.017414 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-hpa-a483d-predictor-serving-cert\"" Jul 09 18:16:30.017605 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.017589 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-hpa-a483d-kube-rbac-proxy-sar-config\"" Jul 09 18:16:30.019335 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.019312 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c9ecdc75-bea4-4175-a028-480693b1a64c-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.019432 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.019363 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c9ecdc75-bea4-4175-a028-480693b1a64c-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.019486 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.019443 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c9ecdc75-bea4-4175-a028-480693b1a64c-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.019639 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.019482 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-274gj\" (UniqueName: \"kubernetes.io/projected/c9ecdc75-bea4-4175-a028-480693b1a64c-kube-api-access-274gj\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.019639 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.019514 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c9ecdc75-bea4-4175-a028-480693b1a64c-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.027507 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.027488 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq"] Jul 09 18:16:30.035989 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.035963 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:16:30.120914 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.120883 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-274gj\" (UniqueName: \"kubernetes.io/projected/c9ecdc75-bea4-4175-a028-480693b1a64c-kube-api-access-274gj\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.121077 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.120930 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c9ecdc75-bea4-4175-a028-480693b1a64c-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.121077 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:16:30.121044 2569 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-serving-cert: secret "isvc-xgboost-graph-raw-hpa-a483d-predictor-serving-cert" not found Jul 09 18:16:30.121194 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:16:30.121110 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9ecdc75-bea4-4175-a028-480693b1a64c-proxy-tls podName:c9ecdc75-bea4-4175-a028-480693b1a64c nodeName:}" failed. No retries permitted until 2026-07-09 18:16:30.621089284 +0000 UTC m=+747.086611673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/c9ecdc75-bea4-4175-a028-480693b1a64c-proxy-tls") pod "isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" (UID: "c9ecdc75-bea4-4175-a028-480693b1a64c") : secret "isvc-xgboost-graph-raw-hpa-a483d-predictor-serving-cert" not found Jul 09 18:16:30.121194 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.121168 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c9ecdc75-bea4-4175-a028-480693b1a64c-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.121308 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.121228 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c9ecdc75-bea4-4175-a028-480693b1a64c-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.121308 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.121290 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c9ecdc75-bea4-4175-a028-480693b1a64c-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.121732 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.121708 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c9ecdc75-bea4-4175-a028-480693b1a64c-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.121940 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.121921 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c9ecdc75-bea4-4175-a028-480693b1a64c-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.122057 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.122035 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c9ecdc75-bea4-4175-a028-480693b1a64c-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.129585 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.129558 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-274gj\" (UniqueName: \"kubernetes.io/projected/c9ecdc75-bea4-4175-a028-480693b1a64c-kube-api-access-274gj\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.135416 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.135393 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:16:30.238477 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.238385 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.24:8643/healthz\": dial tcp 10.134.0.24:8643: connect: connection refused" Jul 09 18:16:30.277613 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.277583 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:16:30.581682 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.581645 2569 generic.go:356] "Generic (PLEG): container finished" podID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerID="6a03ee713d1e94c53d3781f6b67479f08fe4a84ea42b19e36671b087572cce80" exitCode=2 Jul 09 18:16:30.581893 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.581724 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" event={"ID":"9805b5d0-6083-495b-aacf-7396ef8dc11d","Type":"ContainerDied","Data":"6a03ee713d1e94c53d3781f6b67479f08fe4a84ea42b19e36671b087572cce80"} Jul 09 18:16:30.627189 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.627154 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c9ecdc75-bea4-4175-a028-480693b1a64c-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.629674 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.629656 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c9ecdc75-bea4-4175-a028-480693b1a64c-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:30.925810 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:30.925719 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:31.061612 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:31.061580 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq"] Jul 09 18:16:31.063877 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:16:31.063848 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9ecdc75_bea4_4175_a028_480693b1a64c.slice/crio-f2e282f883172f91a53dfcac23ebfef3f7c7a1280f700511091c1003fb8235db WatchSource:0}: Error finding container f2e282f883172f91a53dfcac23ebfef3f7c7a1280f700511091c1003fb8235db: Status 404 returned error can't find the container with id f2e282f883172f91a53dfcac23ebfef3f7c7a1280f700511091c1003fb8235db Jul 09 18:16:31.588082 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:31.588046 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" event={"ID":"c9ecdc75-bea4-4175-a028-480693b1a64c","Type":"ContainerStarted","Data":"77c47f6bc37f95ed96632e247e3915c9477aba2e09ab4e49bd392993208cc9f5"} Jul 09 18:16:31.588301 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:31.588110 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" event={"ID":"c9ecdc75-bea4-4175-a028-480693b1a64c","Type":"ContainerStarted","Data":"f2e282f883172f91a53dfcac23ebfef3f7c7a1280f700511091c1003fb8235db"} Jul 09 18:16:33.843867 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:33.843846 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:16:33.956622 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:33.956533 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9805b5d0-6083-495b-aacf-7396ef8dc11d-cabundle-cert\") pod \"9805b5d0-6083-495b-aacf-7396ef8dc11d\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " Jul 09 18:16:33.956622 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:33.956604 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9805b5d0-6083-495b-aacf-7396ef8dc11d-kserve-provision-location\") pod \"9805b5d0-6083-495b-aacf-7396ef8dc11d\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " Jul 09 18:16:33.956885 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:33.956647 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9stl\" (UniqueName: \"kubernetes.io/projected/9805b5d0-6083-495b-aacf-7396ef8dc11d-kube-api-access-p9stl\") pod \"9805b5d0-6083-495b-aacf-7396ef8dc11d\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " Jul 09 18:16:33.956885 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:33.956675 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9805b5d0-6083-495b-aacf-7396ef8dc11d-kube-rbac-proxy-sar-config\") pod \"9805b5d0-6083-495b-aacf-7396ef8dc11d\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " Jul 09 18:16:33.956885 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:33.956718 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9805b5d0-6083-495b-aacf-7396ef8dc11d-proxy-tls\") pod \"9805b5d0-6083-495b-aacf-7396ef8dc11d\" (UID: \"9805b5d0-6083-495b-aacf-7396ef8dc11d\") " Jul 09 18:16:33.957032 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:33.956961 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9805b5d0-6083-495b-aacf-7396ef8dc11d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "9805b5d0-6083-495b-aacf-7396ef8dc11d" (UID: "9805b5d0-6083-495b-aacf-7396ef8dc11d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:16:33.957086 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:33.957024 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9805b5d0-6083-495b-aacf-7396ef8dc11d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "9805b5d0-6083-495b-aacf-7396ef8dc11d" (UID: "9805b5d0-6083-495b-aacf-7396ef8dc11d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:16:33.957086 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:33.957036 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9805b5d0-6083-495b-aacf-7396ef8dc11d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "9805b5d0-6083-495b-aacf-7396ef8dc11d" (UID: "9805b5d0-6083-495b-aacf-7396ef8dc11d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:16:33.959050 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:33.959022 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9805b5d0-6083-495b-aacf-7396ef8dc11d-kube-api-access-p9stl" (OuterVolumeSpecName: "kube-api-access-p9stl") pod "9805b5d0-6083-495b-aacf-7396ef8dc11d" (UID: "9805b5d0-6083-495b-aacf-7396ef8dc11d"). InnerVolumeSpecName "kube-api-access-p9stl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:16:33.959174 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:33.959097 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9805b5d0-6083-495b-aacf-7396ef8dc11d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9805b5d0-6083-495b-aacf-7396ef8dc11d" (UID: "9805b5d0-6083-495b-aacf-7396ef8dc11d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:16:34.057405 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.057372 2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9805b5d0-6083-495b-aacf-7396ef8dc11d-kserve-provision-location\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:16:34.057405 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.057399 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9stl\" (UniqueName: \"kubernetes.io/projected/9805b5d0-6083-495b-aacf-7396ef8dc11d-kube-api-access-p9stl\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:16:34.057405 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.057409 2569 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9805b5d0-6083-495b-aacf-7396ef8dc11d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:16:34.057614 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.057418 2569 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9805b5d0-6083-495b-aacf-7396ef8dc11d-proxy-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:16:34.057614 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.057427 2569 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9805b5d0-6083-495b-aacf-7396ef8dc11d-cabundle-cert\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:16:34.610595 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.610558 2569 generic.go:356] "Generic (PLEG): container finished" podID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerID="d7367a0055de286c2750a2239753577f1bd840308b3786033f6ad72a0723d6d8" exitCode=0 Jul 09 18:16:34.610822 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.610625 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" Jul 09 18:16:34.610822 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.610671 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" event={"ID":"9805b5d0-6083-495b-aacf-7396ef8dc11d","Type":"ContainerDied","Data":"d7367a0055de286c2750a2239753577f1bd840308b3786033f6ad72a0723d6d8"} Jul 09 18:16:34.610822 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.610759 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x" event={"ID":"9805b5d0-6083-495b-aacf-7396ef8dc11d","Type":"ContainerDied","Data":"fef0882ea7452b876dbfd478253b45c80ff56ea45b5cd02c22c70eca279ac680"} Jul 09 18:16:34.610822 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.610788 2569 scope.go:117] "RemoveContainer" containerID="6a03ee713d1e94c53d3781f6b67479f08fe4a84ea42b19e36671b087572cce80" Jul 09 18:16:34.619565 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.619542 2569 scope.go:117] "RemoveContainer" containerID="d7367a0055de286c2750a2239753577f1bd840308b3786033f6ad72a0723d6d8" Jul 09 18:16:34.627055 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.627038 2569 scope.go:117] "RemoveContainer" containerID="8ca3f5f06356664fc9ec57cffa5fe8120b3ad44c8677afba84a688a8fff1f6b2" Jul 09 18:16:34.634211 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.634187 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x"] Jul 09 18:16:34.635179 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.635166 2569 scope.go:117] "RemoveContainer" containerID="6a03ee713d1e94c53d3781f6b67479f08fe4a84ea42b19e36671b087572cce80" Jul 09 18:16:34.635421 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:16:34.635402 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a03ee713d1e94c53d3781f6b67479f08fe4a84ea42b19e36671b087572cce80\": container with ID starting with 6a03ee713d1e94c53d3781f6b67479f08fe4a84ea42b19e36671b087572cce80 not found: ID does not exist" containerID="6a03ee713d1e94c53d3781f6b67479f08fe4a84ea42b19e36671b087572cce80" Jul 09 18:16:34.635471 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.635429 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a03ee713d1e94c53d3781f6b67479f08fe4a84ea42b19e36671b087572cce80"} err="failed to get container status \"6a03ee713d1e94c53d3781f6b67479f08fe4a84ea42b19e36671b087572cce80\": rpc error: code = NotFound desc = could not find container \"6a03ee713d1e94c53d3781f6b67479f08fe4a84ea42b19e36671b087572cce80\": container with ID starting with 6a03ee713d1e94c53d3781f6b67479f08fe4a84ea42b19e36671b087572cce80 not found: ID does not exist" Jul 09 18:16:34.635471 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.635446 2569 scope.go:117] "RemoveContainer" containerID="d7367a0055de286c2750a2239753577f1bd840308b3786033f6ad72a0723d6d8" Jul 09 18:16:34.635748 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:16:34.635732 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7367a0055de286c2750a2239753577f1bd840308b3786033f6ad72a0723d6d8\": container with ID starting with d7367a0055de286c2750a2239753577f1bd840308b3786033f6ad72a0723d6d8 not found: ID does not exist" containerID="d7367a0055de286c2750a2239753577f1bd840308b3786033f6ad72a0723d6d8" Jul 09 18:16:34.635797 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.635754 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7367a0055de286c2750a2239753577f1bd840308b3786033f6ad72a0723d6d8"} err="failed to get container status \"d7367a0055de286c2750a2239753577f1bd840308b3786033f6ad72a0723d6d8\": rpc error: code = NotFound desc = could not find container \"d7367a0055de286c2750a2239753577f1bd840308b3786033f6ad72a0723d6d8\": container with ID starting with d7367a0055de286c2750a2239753577f1bd840308b3786033f6ad72a0723d6d8 not found: ID does not exist" Jul 09 18:16:34.635797 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.635768 2569 scope.go:117] "RemoveContainer" containerID="8ca3f5f06356664fc9ec57cffa5fe8120b3ad44c8677afba84a688a8fff1f6b2" Jul 09 18:16:34.636054 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:16:34.636029 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ca3f5f06356664fc9ec57cffa5fe8120b3ad44c8677afba84a688a8fff1f6b2\": container with ID starting with 8ca3f5f06356664fc9ec57cffa5fe8120b3ad44c8677afba84a688a8fff1f6b2 not found: ID does not exist" containerID="8ca3f5f06356664fc9ec57cffa5fe8120b3ad44c8677afba84a688a8fff1f6b2" Jul 09 18:16:34.636136 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.636063 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ca3f5f06356664fc9ec57cffa5fe8120b3ad44c8677afba84a688a8fff1f6b2"} err="failed to get container status \"8ca3f5f06356664fc9ec57cffa5fe8120b3ad44c8677afba84a688a8fff1f6b2\": rpc error: code = NotFound desc = could not find container \"8ca3f5f06356664fc9ec57cffa5fe8120b3ad44c8677afba84a688a8fff1f6b2\": container with ID starting with 8ca3f5f06356664fc9ec57cffa5fe8120b3ad44c8677afba84a688a8fff1f6b2 not found: ID does not exist" Jul 09 18:16:34.638443 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:34.638425 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-a2b25-predictor-7c7f4677bf-w9s8x"] Jul 09 18:16:35.616595 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:35.616562 2569 generic.go:356] "Generic (PLEG): container finished" podID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerID="77c47f6bc37f95ed96632e247e3915c9477aba2e09ab4e49bd392993208cc9f5" exitCode=0 Jul 09 18:16:35.617101 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:35.616636 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" event={"ID":"c9ecdc75-bea4-4175-a028-480693b1a64c","Type":"ContainerDied","Data":"77c47f6bc37f95ed96632e247e3915c9477aba2e09ab4e49bd392993208cc9f5"} Jul 09 18:16:36.196821 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:36.196786 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" path="/var/lib/kubelet/pods/9805b5d0-6083-495b-aacf-7396ef8dc11d/volumes" Jul 09 18:16:36.624847 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:36.624806 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" event={"ID":"c9ecdc75-bea4-4175-a028-480693b1a64c","Type":"ContainerStarted","Data":"30c9b4e527e2e6e22149ce50ea5f1f77c36cb3ee68aa7d19c49183d708f7c718"} Jul 09 18:16:36.625283 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:36.624875 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" event={"ID":"c9ecdc75-bea4-4175-a028-480693b1a64c","Type":"ContainerStarted","Data":"033be881124d43c4fc02b99c8b8388b3b2dc94ab167194b1aea31d8130a2d3b2"} Jul 09 18:16:36.625283 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:36.625157 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:36.625386 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:36.625302 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:36.626715 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:36.626670 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Jul 09 18:16:36.645805 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:36.645747 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" podStartSLOduration=3.728918865 podStartE2EDuration="7.645726908s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.916808043s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:16:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:16:36.644218771 +0000 UTC m=+753.109741229" watchObservedRunningTime="2026-07-09 18:16:36.645726908 +0000 UTC m=+753.111249319" Jul 09 18:16:37.629353 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:37.629307 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Jul 09 18:16:42.634388 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:42.634356 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:16:42.634913 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:42.634886 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Jul 09 18:16:52.635885 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:16:52.635839 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Jul 09 18:17:02.635708 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:17:02.635633 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Jul 09 18:17:12.634801 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:17:12.634760 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Jul 09 18:17:22.635831 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:17:22.635791 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Jul 09 18:17:29.300886 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:17:29.300808 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:17:29.465475 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:17:29.465447 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:17:32.634912 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:17:32.634877 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Jul 09 18:17:42.635864 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:17:42.635825 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:18:09.976339 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:09.976307 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:18:10.145628 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:10.145597 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:18:20.054787 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:20.054752 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:18:20.251909 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:20.251875 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:18:20.356199 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:20.356109 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:18:20.365293 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:20.365260 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq"] Jul 09 18:18:20.365554 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:20.365531 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kserve-container" containerID="cri-o://033be881124d43c4fc02b99c8b8388b3b2dc94ab167194b1aea31d8130a2d3b2" gracePeriod=30 Jul 09 18:18:20.365652 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:20.365532 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kube-rbac-proxy" containerID="cri-o://30c9b4e527e2e6e22149ce50ea5f1f77c36cb3ee68aa7d19c49183d708f7c718" gracePeriod=30 Jul 09 18:18:20.447286 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:20.447249 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:18:21.076450 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:21.076418 2569 generic.go:356] "Generic (PLEG): container finished" podID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerID="30c9b4e527e2e6e22149ce50ea5f1f77c36cb3ee68aa7d19c49183d708f7c718" exitCode=2 Jul 09 18:18:21.076833 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:21.076492 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" event={"ID":"c9ecdc75-bea4-4175-a028-480693b1a64c","Type":"ContainerDied","Data":"30c9b4e527e2e6e22149ce50ea5f1f77c36cb3ee68aa7d19c49183d708f7c718"} Jul 09 18:18:22.629772 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:22.629733 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.25:8643/healthz\": dial tcp 10.134.0.25:8643: connect: connection refused" Jul 09 18:18:22.635305 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:22.635280 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Jul 09 18:18:23.910951 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:23.910928 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:18:24.023550 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.023523 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c9ecdc75-bea4-4175-a028-480693b1a64c-kube-rbac-proxy-sar-config\") pod \"c9ecdc75-bea4-4175-a028-480693b1a64c\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " Jul 09 18:18:24.023741 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.023581 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-274gj\" (UniqueName: \"kubernetes.io/projected/c9ecdc75-bea4-4175-a028-480693b1a64c-kube-api-access-274gj\") pod \"c9ecdc75-bea4-4175-a028-480693b1a64c\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " Jul 09 18:18:24.023741 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.023630 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c9ecdc75-bea4-4175-a028-480693b1a64c-kserve-provision-location\") pod \"c9ecdc75-bea4-4175-a028-480693b1a64c\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " Jul 09 18:18:24.023741 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.023663 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c9ecdc75-bea4-4175-a028-480693b1a64c-proxy-tls\") pod \"c9ecdc75-bea4-4175-a028-480693b1a64c\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " Jul 09 18:18:24.023741 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.023726 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c9ecdc75-bea4-4175-a028-480693b1a64c-cabundle-cert\") pod \"c9ecdc75-bea4-4175-a028-480693b1a64c\" (UID: \"c9ecdc75-bea4-4175-a028-480693b1a64c\") " Jul 09 18:18:24.024040 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.024011 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9ecdc75-bea4-4175-a028-480693b1a64c-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "c9ecdc75-bea4-4175-a028-480693b1a64c" (UID: "c9ecdc75-bea4-4175-a028-480693b1a64c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:18:24.024096 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.023957 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9ecdc75-bea4-4175-a028-480693b1a64c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c9ecdc75-bea4-4175-a028-480693b1a64c" (UID: "c9ecdc75-bea4-4175-a028-480693b1a64c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:18:24.024096 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.024078 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9ecdc75-bea4-4175-a028-480693b1a64c-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "c9ecdc75-bea4-4175-a028-480693b1a64c" (UID: "c9ecdc75-bea4-4175-a028-480693b1a64c"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 18:18:24.025946 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.025921 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9ecdc75-bea4-4175-a028-480693b1a64c-kube-api-access-274gj" (OuterVolumeSpecName: "kube-api-access-274gj") pod "c9ecdc75-bea4-4175-a028-480693b1a64c" (UID: "c9ecdc75-bea4-4175-a028-480693b1a64c"). InnerVolumeSpecName "kube-api-access-274gj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:18:24.025946 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.025932 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9ecdc75-bea4-4175-a028-480693b1a64c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c9ecdc75-bea4-4175-a028-480693b1a64c" (UID: "c9ecdc75-bea4-4175-a028-480693b1a64c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 18:18:24.091078 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.091047 2569 generic.go:356] "Generic (PLEG): container finished" podID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerID="033be881124d43c4fc02b99c8b8388b3b2dc94ab167194b1aea31d8130a2d3b2" exitCode=0 Jul 09 18:18:24.091207 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.091110 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" Jul 09 18:18:24.091207 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.091136 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" event={"ID":"c9ecdc75-bea4-4175-a028-480693b1a64c","Type":"ContainerDied","Data":"033be881124d43c4fc02b99c8b8388b3b2dc94ab167194b1aea31d8130a2d3b2"} Jul 09 18:18:24.091207 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.091203 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq" event={"ID":"c9ecdc75-bea4-4175-a028-480693b1a64c","Type":"ContainerDied","Data":"f2e282f883172f91a53dfcac23ebfef3f7c7a1280f700511091c1003fb8235db"} Jul 09 18:18:24.091303 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.091240 2569 scope.go:117] "RemoveContainer" containerID="30c9b4e527e2e6e22149ce50ea5f1f77c36cb3ee68aa7d19c49183d708f7c718" Jul 09 18:18:24.100060 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.100038 2569 scope.go:117] "RemoveContainer" containerID="033be881124d43c4fc02b99c8b8388b3b2dc94ab167194b1aea31d8130a2d3b2" Jul 09 18:18:24.107549 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.107530 2569 scope.go:117] "RemoveContainer" containerID="77c47f6bc37f95ed96632e247e3915c9477aba2e09ab4e49bd392993208cc9f5" Jul 09 18:18:24.115270 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.115251 2569 scope.go:117] "RemoveContainer" containerID="30c9b4e527e2e6e22149ce50ea5f1f77c36cb3ee68aa7d19c49183d708f7c718" Jul 09 18:18:24.115569 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:18:24.115548 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30c9b4e527e2e6e22149ce50ea5f1f77c36cb3ee68aa7d19c49183d708f7c718\": container with ID starting with 30c9b4e527e2e6e22149ce50ea5f1f77c36cb3ee68aa7d19c49183d708f7c718 not found: ID does not exist" containerID="30c9b4e527e2e6e22149ce50ea5f1f77c36cb3ee68aa7d19c49183d708f7c718" Jul 09 18:18:24.115641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.115577 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30c9b4e527e2e6e22149ce50ea5f1f77c36cb3ee68aa7d19c49183d708f7c718"} err="failed to get container status \"30c9b4e527e2e6e22149ce50ea5f1f77c36cb3ee68aa7d19c49183d708f7c718\": rpc error: code = NotFound desc = could not find container \"30c9b4e527e2e6e22149ce50ea5f1f77c36cb3ee68aa7d19c49183d708f7c718\": container with ID starting with 30c9b4e527e2e6e22149ce50ea5f1f77c36cb3ee68aa7d19c49183d708f7c718 not found: ID does not exist" Jul 09 18:18:24.115641 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.115596 2569 scope.go:117] "RemoveContainer" containerID="033be881124d43c4fc02b99c8b8388b3b2dc94ab167194b1aea31d8130a2d3b2" Jul 09 18:18:24.115839 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.115818 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq"] Jul 09 18:18:24.115887 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:18:24.115844 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"033be881124d43c4fc02b99c8b8388b3b2dc94ab167194b1aea31d8130a2d3b2\": container with ID starting with 033be881124d43c4fc02b99c8b8388b3b2dc94ab167194b1aea31d8130a2d3b2 not found: ID does not exist" containerID="033be881124d43c4fc02b99c8b8388b3b2dc94ab167194b1aea31d8130a2d3b2" Jul 09 18:18:24.115887 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.115864 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"033be881124d43c4fc02b99c8b8388b3b2dc94ab167194b1aea31d8130a2d3b2"} err="failed to get container status \"033be881124d43c4fc02b99c8b8388b3b2dc94ab167194b1aea31d8130a2d3b2\": rpc error: code = NotFound desc = could not find container \"033be881124d43c4fc02b99c8b8388b3b2dc94ab167194b1aea31d8130a2d3b2\": container with ID starting with 033be881124d43c4fc02b99c8b8388b3b2dc94ab167194b1aea31d8130a2d3b2 not found: ID does not exist" Jul 09 18:18:24.115887 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.115881 2569 scope.go:117] "RemoveContainer" containerID="77c47f6bc37f95ed96632e247e3915c9477aba2e09ab4e49bd392993208cc9f5" Jul 09 18:18:24.116110 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:18:24.116093 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77c47f6bc37f95ed96632e247e3915c9477aba2e09ab4e49bd392993208cc9f5\": container with ID starting with 77c47f6bc37f95ed96632e247e3915c9477aba2e09ab4e49bd392993208cc9f5 not found: ID does not exist" containerID="77c47f6bc37f95ed96632e247e3915c9477aba2e09ab4e49bd392993208cc9f5" Jul 09 18:18:24.116164 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.116119 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77c47f6bc37f95ed96632e247e3915c9477aba2e09ab4e49bd392993208cc9f5"} err="failed to get container status \"77c47f6bc37f95ed96632e247e3915c9477aba2e09ab4e49bd392993208cc9f5\": rpc error: code = NotFound desc = could not find container \"77c47f6bc37f95ed96632e247e3915c9477aba2e09ab4e49bd392993208cc9f5\": container with ID starting with 77c47f6bc37f95ed96632e247e3915c9477aba2e09ab4e49bd392993208cc9f5 not found: ID does not exist" Jul 09 18:18:24.121798 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.121771 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-hpa-a483d-predictor-b6c584fd5-fzbnq"] Jul 09 18:18:24.125791 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.125763 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-274gj\" (UniqueName: \"kubernetes.io/projected/c9ecdc75-bea4-4175-a028-480693b1a64c-kube-api-access-274gj\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:18:24.125791 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.125785 2569 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c9ecdc75-bea4-4175-a028-480693b1a64c-kserve-provision-location\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:18:24.125791 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.125795 2569 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c9ecdc75-bea4-4175-a028-480693b1a64c-proxy-tls\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:18:24.126011 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.125805 2569 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c9ecdc75-bea4-4175-a028-480693b1a64c-cabundle-cert\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:18:24.126011 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.125813 2569 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c9ecdc75-bea4-4175-a028-480693b1a64c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:18:24.196317 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:24.196291 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" path="/var/lib/kubelet/pods/c9ecdc75-bea4-4175-a028-480693b1a64c/volumes" Jul 09 18:18:30.251994 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:30.251956 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:18:30.874037 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:18:30.874005 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:20:05.647252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:20:05.647202 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:20:07.104220 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:20:07.104174 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:20:17.602408 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:20:17.602370 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:21:30.520138 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:21:30.520097 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:22:55.862458 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:22:55.862424 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:23:00.960651 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:23:00.960613 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:23:01.051446 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:23:01.051414 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:24:31.204092 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:24:31.204056 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:24:31.425414 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:24:31.425386 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:24:49.290957 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:24:49.290925 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:24:49.364050 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:24:49.364017 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:24:54.475225 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:24:54.475124 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:24:55.109829 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:24:55.109794 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:25:04.556147 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:25:04.556100 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:25:09.744618 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:25:09.744579 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:25:11.125817 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:25:11.125787 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:26:40.046346 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:26:40.046265 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:26:45.068854 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:26:45.068818 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:26:45.266529 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:26:45.266492 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:28:10.641749 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:10.641715 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:28:15.238661 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:15.238619 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:28:17.095698 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:17.095660 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:28:21.640157 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640122 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-55gf2/must-gather-jzfkx"] Jul 09 18:28:21.640543 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640501 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="storage-initializer" Jul 09 18:28:21.640543 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640512 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="storage-initializer" Jul 09 18:28:21.640543 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640524 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kserve-container" Jul 09 18:28:21.640543 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640530 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kserve-container" Jul 09 18:28:21.640543 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640540 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kube-rbac-proxy" Jul 09 18:28:21.640543 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640545 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kube-rbac-proxy" Jul 09 18:28:21.640789 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640559 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kube-rbac-proxy" Jul 09 18:28:21.640789 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640565 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kube-rbac-proxy" Jul 09 18:28:21.640789 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640576 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="storage-initializer" Jul 09 18:28:21.640789 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640582 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="storage-initializer" Jul 09 18:28:21.640789 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640588 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kserve-container" Jul 09 18:28:21.640789 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640593 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kserve-container" Jul 09 18:28:21.640789 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640657 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kube-rbac-proxy" Jul 09 18:28:21.640789 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640665 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9ecdc75-bea4-4175-a028-480693b1a64c" containerName="kserve-container" Jul 09 18:28:21.640789 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640674 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kube-rbac-proxy" Jul 09 18:28:21.640789 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.640680 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9805b5d0-6083-495b-aacf-7396ef8dc11d" containerName="kserve-container" Jul 09 18:28:21.643831 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.643808 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-55gf2/must-gather-jzfkx" Jul 09 18:28:21.650120 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.650089 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-55gf2\"/\"default-dockercfg-fkvhp\"" Jul 09 18:28:21.650739 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.650721 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-55gf2\"/\"kube-root-ca.crt\"" Jul 09 18:28:21.650847 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.650777 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-55gf2\"/\"openshift-service-ca.crt\"" Jul 09 18:28:21.670635 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.670601 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-55gf2/must-gather-jzfkx"] Jul 09 18:28:21.686666 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.686616 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:28:21.778288 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.778248 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz7wb\" (UniqueName: \"kubernetes.io/projected/81caf98e-6196-4f2b-a37e-4744b0e13825-kube-api-access-fz7wb\") pod \"must-gather-jzfkx\" (UID: \"81caf98e-6196-4f2b-a37e-4744b0e13825\") " pod="openshift-must-gather-55gf2/must-gather-jzfkx" Jul 09 18:28:21.778500 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.778313 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/81caf98e-6196-4f2b-a37e-4744b0e13825-must-gather-output\") pod \"must-gather-jzfkx\" (UID: \"81caf98e-6196-4f2b-a37e-4744b0e13825\") " pod="openshift-must-gather-55gf2/must-gather-jzfkx" Jul 09 18:28:21.782395 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.782371 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:28:21.879513 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.879478 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz7wb\" (UniqueName: \"kubernetes.io/projected/81caf98e-6196-4f2b-a37e-4744b0e13825-kube-api-access-fz7wb\") pod \"must-gather-jzfkx\" (UID: \"81caf98e-6196-4f2b-a37e-4744b0e13825\") " pod="openshift-must-gather-55gf2/must-gather-jzfkx" Jul 09 18:28:21.879678 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.879541 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/81caf98e-6196-4f2b-a37e-4744b0e13825-must-gather-output\") pod \"must-gather-jzfkx\" (UID: \"81caf98e-6196-4f2b-a37e-4744b0e13825\") " pod="openshift-must-gather-55gf2/must-gather-jzfkx" Jul 09 18:28:21.879875 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.879859 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/81caf98e-6196-4f2b-a37e-4744b0e13825-must-gather-output\") pod \"must-gather-jzfkx\" (UID: \"81caf98e-6196-4f2b-a37e-4744b0e13825\") " pod="openshift-must-gather-55gf2/must-gather-jzfkx" Jul 09 18:28:21.889252 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.889220 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz7wb\" (UniqueName: \"kubernetes.io/projected/81caf98e-6196-4f2b-a37e-4744b0e13825-kube-api-access-fz7wb\") pod \"must-gather-jzfkx\" (UID: \"81caf98e-6196-4f2b-a37e-4744b0e13825\") " pod="openshift-must-gather-55gf2/must-gather-jzfkx" Jul 09 18:28:21.968440 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:21.968337 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-55gf2/must-gather-jzfkx" Jul 09 18:28:22.114894 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:22.114867 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-55gf2/must-gather-jzfkx"] Jul 09 18:28:22.117213 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:28:22.117181 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81caf98e_6196_4f2b_a37e_4744b0e13825.slice/crio-de9c9f4e4336374200bcdbed399d768c2e132b05939bf3a55436f651d23f6568 WatchSource:0}: Error finding container de9c9f4e4336374200bcdbed399d768c2e132b05939bf3a55436f651d23f6568: Status 404 returned error can't find the container with id de9c9f4e4336374200bcdbed399d768c2e132b05939bf3a55436f651d23f6568 Jul 09 18:28:22.119268 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:22.119244 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 18:28:22.608765 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:22.608730 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-55gf2/must-gather-jzfkx" event={"ID":"81caf98e-6196-4f2b-a37e-4744b0e13825","Type":"ContainerStarted","Data":"de9c9f4e4336374200bcdbed399d768c2e132b05939bf3a55436f651d23f6568"} Jul 09 18:28:28.645232 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:28.645198 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-55gf2/must-gather-jzfkx" event={"ID":"81caf98e-6196-4f2b-a37e-4744b0e13825","Type":"ContainerStarted","Data":"9f8eb32f09d3d88e8da56dabb63b8c2b9be940bb3b339883e2370ccb07bb48a8"} Jul 09 18:28:28.645232 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:28.645243 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-55gf2/must-gather-jzfkx" event={"ID":"81caf98e-6196-4f2b-a37e-4744b0e13825","Type":"ContainerStarted","Data":"3a5af1495b986cc58341bed1b89bf02a20c9e5a71c47089809f5c78f937d8261"} Jul 09 18:28:28.676821 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:28.676763 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-55gf2/must-gather-jzfkx" podStartSLOduration=1.449256921 podStartE2EDuration="7.676743587s" totalImagesPullingTime="6.227486666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:28:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:28:28.675810413 +0000 UTC m=+1465.141332825" watchObservedRunningTime="2026-07-09 18:28:28.676743587 +0000 UTC m=+1465.142265998" Jul 09 18:28:51.759603 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:51.759571 2569 generic.go:356] "Generic (PLEG): container finished" podID="81caf98e-6196-4f2b-a37e-4744b0e13825" containerID="3a5af1495b986cc58341bed1b89bf02a20c9e5a71c47089809f5c78f937d8261" exitCode=0 Jul 09 18:28:51.760035 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:51.759645 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-55gf2/must-gather-jzfkx" event={"ID":"81caf98e-6196-4f2b-a37e-4744b0e13825","Type":"ContainerDied","Data":"3a5af1495b986cc58341bed1b89bf02a20c9e5a71c47089809f5c78f937d8261"} Jul 09 18:28:51.760035 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:51.760023 2569 scope.go:117] "RemoveContainer" containerID="3a5af1495b986cc58341bed1b89bf02a20c9e5a71c47089809f5c78f937d8261" Jul 09 18:28:52.347301 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:52.347276 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-55gf2_must-gather-jzfkx_81caf98e-6196-4f2b-a37e-4744b0e13825/gather/0.log" Jul 09 18:28:53.504731 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:53.504666 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:28:53.603314 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:53.603283 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:28:55.656495 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:55.656462 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sq92v_e1661ba5-6bd9-484b-97a0-8b28cb87e456/global-pull-secret-syncer/0.log" Jul 09 18:28:55.768188 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:55.768157 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" Jul 09 18:28:55.922144 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:55.922063 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-168.ec2.internal_631442f90a9aec34b0d279ea0f5f9c73/haproxy/0.log" Jul 09 18:28:57.790437 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:57.790407 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-55gf2/must-gather-jzfkx"] Jul 09 18:28:57.790995 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:57.790678 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-55gf2/must-gather-jzfkx" podUID="81caf98e-6196-4f2b-a37e-4744b0e13825" containerName="copy" containerID="cri-o://9f8eb32f09d3d88e8da56dabb63b8c2b9be940bb3b339883e2370ccb07bb48a8" gracePeriod=2 Jul 09 18:28:57.793068 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:28:57.793036 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-jzfkx\" is forbidden: User \"system:node:ip-10-0-137-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-55gf2\": no relationship found between node 'ip-10-0-137-168.ec2.internal' and this object" podUID="81caf98e-6196-4f2b-a37e-4744b0e13825" pod="openshift-must-gather-55gf2/must-gather-jzfkx" Jul 09 18:28:57.793276 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:57.793256 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-55gf2/must-gather-jzfkx"] Jul 09 18:28:58.025894 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.025873 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-55gf2_must-gather-jzfkx_81caf98e-6196-4f2b-a37e-4744b0e13825/copy/0.log" Jul 09 18:28:58.027466 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.027448 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-55gf2/must-gather-jzfkx" Jul 09 18:28:58.029488 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:28:58.029462 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-jzfkx\" is forbidden: User \"system:node:ip-10-0-137-168.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-55gf2\": no relationship found between node 'ip-10-0-137-168.ec2.internal' and this object" podUID="81caf98e-6196-4f2b-a37e-4744b0e13825" pod="openshift-must-gather-55gf2/must-gather-jzfkx" Jul 09 18:28:58.124059 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.123981 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/81caf98e-6196-4f2b-a37e-4744b0e13825-must-gather-output\") pod \"81caf98e-6196-4f2b-a37e-4744b0e13825\" (UID: \"81caf98e-6196-4f2b-a37e-4744b0e13825\") " Jul 09 18:28:58.124212 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.124094 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fz7wb\" (UniqueName: \"kubernetes.io/projected/81caf98e-6196-4f2b-a37e-4744b0e13825-kube-api-access-fz7wb\") pod \"81caf98e-6196-4f2b-a37e-4744b0e13825\" (UID: \"81caf98e-6196-4f2b-a37e-4744b0e13825\") " Jul 09 18:28:58.125476 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.125441 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81caf98e-6196-4f2b-a37e-4744b0e13825-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "81caf98e-6196-4f2b-a37e-4744b0e13825" (UID: "81caf98e-6196-4f2b-a37e-4744b0e13825"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 18:28:58.126394 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.126370 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81caf98e-6196-4f2b-a37e-4744b0e13825-kube-api-access-fz7wb" (OuterVolumeSpecName: "kube-api-access-fz7wb") pod "81caf98e-6196-4f2b-a37e-4744b0e13825" (UID: "81caf98e-6196-4f2b-a37e-4744b0e13825"). InnerVolumeSpecName "kube-api-access-fz7wb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 18:28:58.196751 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.196709 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81caf98e-6196-4f2b-a37e-4744b0e13825" path="/var/lib/kubelet/pods/81caf98e-6196-4f2b-a37e-4744b0e13825/volumes" Jul 09 18:28:58.225055 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.225031 2569 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/81caf98e-6196-4f2b-a37e-4744b0e13825-must-gather-output\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:28:58.225055 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.225055 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fz7wb\" (UniqueName: \"kubernetes.io/projected/81caf98e-6196-4f2b-a37e-4744b0e13825-kube-api-access-fz7wb\") on node \"ip-10-0-137-168.ec2.internal\" DevicePath \"\"" Jul 09 18:28:58.790073 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.790045 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-55gf2_must-gather-jzfkx_81caf98e-6196-4f2b-a37e-4744b0e13825/copy/0.log" Jul 09 18:28:58.790435 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.790410 2569 generic.go:356] "Generic (PLEG): container finished" podID="81caf98e-6196-4f2b-a37e-4744b0e13825" containerID="9f8eb32f09d3d88e8da56dabb63b8c2b9be940bb3b339883e2370ccb07bb48a8" exitCode=143 Jul 09 18:28:58.790536 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.790470 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-55gf2/must-gather-jzfkx" Jul 09 18:28:58.790536 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.790499 2569 scope.go:117] "RemoveContainer" containerID="9f8eb32f09d3d88e8da56dabb63b8c2b9be940bb3b339883e2370ccb07bb48a8" Jul 09 18:28:58.798649 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.798630 2569 scope.go:117] "RemoveContainer" containerID="3a5af1495b986cc58341bed1b89bf02a20c9e5a71c47089809f5c78f937d8261" Jul 09 18:28:58.809773 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.809745 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:28:58.811767 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.811669 2569 scope.go:117] "RemoveContainer" containerID="9f8eb32f09d3d88e8da56dabb63b8c2b9be940bb3b339883e2370ccb07bb48a8" Jul 09 18:28:58.811999 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:28:58.811978 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f8eb32f09d3d88e8da56dabb63b8c2b9be940bb3b339883e2370ccb07bb48a8\": container with ID starting with 9f8eb32f09d3d88e8da56dabb63b8c2b9be940bb3b339883e2370ccb07bb48a8 not found: ID does not exist" containerID="9f8eb32f09d3d88e8da56dabb63b8c2b9be940bb3b339883e2370ccb07bb48a8" Jul 09 18:28:58.812056 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.812007 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f8eb32f09d3d88e8da56dabb63b8c2b9be940bb3b339883e2370ccb07bb48a8"} err="failed to get container status \"9f8eb32f09d3d88e8da56dabb63b8c2b9be940bb3b339883e2370ccb07bb48a8\": rpc error: code = NotFound desc = could not find container \"9f8eb32f09d3d88e8da56dabb63b8c2b9be940bb3b339883e2370ccb07bb48a8\": container with ID starting with 9f8eb32f09d3d88e8da56dabb63b8c2b9be940bb3b339883e2370ccb07bb48a8 not found: ID does not exist" Jul 09 18:28:58.812056 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.812025 2569 scope.go:117] "RemoveContainer" containerID="3a5af1495b986cc58341bed1b89bf02a20c9e5a71c47089809f5c78f937d8261" Jul 09 18:28:58.812317 ip-10-0-137-168 kubenswrapper[2569]: E0709 18:28:58.812297 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a5af1495b986cc58341bed1b89bf02a20c9e5a71c47089809f5c78f937d8261\": container with ID starting with 3a5af1495b986cc58341bed1b89bf02a20c9e5a71c47089809f5c78f937d8261 not found: ID does not exist" containerID="3a5af1495b986cc58341bed1b89bf02a20c9e5a71c47089809f5c78f937d8261" Jul 09 18:28:58.812375 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.812327 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a5af1495b986cc58341bed1b89bf02a20c9e5a71c47089809f5c78f937d8261"} err="failed to get container status \"3a5af1495b986cc58341bed1b89bf02a20c9e5a71c47089809f5c78f937d8261\": rpc error: code = NotFound desc = could not find container \"3a5af1495b986cc58341bed1b89bf02a20c9e5a71c47089809f5c78f937d8261\": container with ID starting with 3a5af1495b986cc58341bed1b89bf02a20c9e5a71c47089809f5c78f937d8261 not found: ID does not exist" Jul 09 18:28:58.898330 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:58.898301 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2qp88_3fa7022d-512b-4625-95bb-1342a17d197f/konnectivity-agent/0.log" limit=1024 Jul 09 18:28:59.490856 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.490825 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d58a497b-9688-427a-b3dd-72c8614bcb2f/alertmanager/0.log" Jul 09 18:28:59.521616 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.521586 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d58a497b-9688-427a-b3dd-72c8614bcb2f/config-reloader/0.log" Jul 09 18:28:59.551557 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.551532 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d58a497b-9688-427a-b3dd-72c8614bcb2f/kube-rbac-proxy-web/0.log" Jul 09 18:28:59.587155 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.587134 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d58a497b-9688-427a-b3dd-72c8614bcb2f/kube-rbac-proxy/0.log" Jul 09 18:28:59.617664 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.617641 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d58a497b-9688-427a-b3dd-72c8614bcb2f/kube-rbac-proxy-metric/0.log" Jul 09 18:28:59.649107 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.649079 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d58a497b-9688-427a-b3dd-72c8614bcb2f/prom-label-proxy/0.log" Jul 09 18:28:59.685546 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.685524 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d58a497b-9688-427a-b3dd-72c8614bcb2f/init-config-reloader/0.log" Jul 09 18:28:59.761958 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.761931 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-7lqhs_37199f74-3a3d-4d96-9b8d-7c248d641d11/kube-state-metrics/0.log" Jul 09 18:28:59.791501 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.791473 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-7lqhs_37199f74-3a3d-4d96-9b8d-7c248d641d11/kube-rbac-proxy-main/0.log" Jul 09 18:28:59.820554 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.820533 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-7lqhs_37199f74-3a3d-4d96-9b8d-7c248d641d11/kube-rbac-proxy-self/0.log" Jul 09 18:28:59.858711 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.858667 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7b8b74fc9f-mpjm7_d91372df-546b-4b7e-9ee1-c2d8d718b727/metrics-server/0.log" Jul 09 18:28:59.890381 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.890357 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-p8sdt_514852e2-de26-4c7f-b071-f875cefae908/monitoring-plugin/0.log" Jul 09 18:28:59.928383 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.928352 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dszrz_bbdd2842-6040-47e2-964f-613c8026aebf/node-exporter/0.log" Jul 09 18:28:59.951544 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.951525 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dszrz_bbdd2842-6040-47e2-964f-613c8026aebf/kube-rbac-proxy/0.log" Jul 09 18:28:59.977321 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:28:59.977295 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dszrz_bbdd2842-6040-47e2-964f-613c8026aebf/init-textfile/0.log" Jul 09 18:29:00.205350 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.205308 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-brlv8_05a00921-ffc8-419d-b9e1-e76d8c6580ee/kube-rbac-proxy-main/0.log" Jul 09 18:29:00.237885 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.237844 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-brlv8_05a00921-ffc8-419d-b9e1-e76d8c6580ee/kube-rbac-proxy-self/0.log" Jul 09 18:29:00.264153 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.264127 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-brlv8_05a00921-ffc8-419d-b9e1-e76d8c6580ee/openshift-state-metrics/0.log" Jul 09 18:29:00.305947 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.305921 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b304d2bb-d80b-4d6d-8886-1d5c1e709e3b/prometheus/0.log" Jul 09 18:29:00.330895 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.330802 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b304d2bb-d80b-4d6d-8886-1d5c1e709e3b/config-reloader/0.log" Jul 09 18:29:00.356349 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.356311 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b304d2bb-d80b-4d6d-8886-1d5c1e709e3b/thanos-sidecar/0.log" Jul 09 18:29:00.385282 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.385248 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b304d2bb-d80b-4d6d-8886-1d5c1e709e3b/kube-rbac-proxy-web/0.log" Jul 09 18:29:00.417224 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.417190 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b304d2bb-d80b-4d6d-8886-1d5c1e709e3b/kube-rbac-proxy/0.log" Jul 09 18:29:00.449136 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.449089 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b304d2bb-d80b-4d6d-8886-1d5c1e709e3b/kube-rbac-proxy-thanos/0.log" Jul 09 18:29:00.523830 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.523805 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b304d2bb-d80b-4d6d-8886-1d5c1e709e3b/init-config-reloader/0.log" Jul 09 18:29:00.573706 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.573666 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-pkjvw_53f1ad17-1807-49b3-87b1-148ea3b3c791/prometheus-operator/0.log" Jul 09 18:29:00.598756 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.598671 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-pkjvw_53f1ad17-1807-49b3-87b1-148ea3b3c791/kube-rbac-proxy/0.log" Jul 09 18:29:00.659890 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.659867 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d88b99897-xdjwj_5bff25a4-76cb-4bb4-92d9-b31ca66ffd71/telemeter-client/0.log" Jul 09 18:29:00.686179 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.686155 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d88b99897-xdjwj_5bff25a4-76cb-4bb4-92d9-b31ca66ffd71/reload/0.log" Jul 09 18:29:00.712468 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.712441 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d88b99897-xdjwj_5bff25a4-76cb-4bb4-92d9-b31ca66ffd71/kube-rbac-proxy/0.log" Jul 09 18:29:00.749987 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.749960 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68c4666c44-5grzf_c332145d-ab9e-4777-aa6f-4ffa8c0a2de0/thanos-query/0.log" Jul 09 18:29:00.789962 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.789936 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68c4666c44-5grzf_c332145d-ab9e-4777-aa6f-4ffa8c0a2de0/kube-rbac-proxy-web/0.log" Jul 09 18:29:00.829948 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.829914 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68c4666c44-5grzf_c332145d-ab9e-4777-aa6f-4ffa8c0a2de0/kube-rbac-proxy/0.log" Jul 09 18:29:00.872241 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.872171 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68c4666c44-5grzf_c332145d-ab9e-4777-aa6f-4ffa8c0a2de0/prom-label-proxy/0.log" Jul 09 18:29:00.901253 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.901219 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68c4666c44-5grzf_c332145d-ab9e-4777-aa6f-4ffa8c0a2de0/kube-rbac-proxy-rules/0.log" Jul 09 18:29:00.937297 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:00.937267 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68c4666c44-5grzf_c332145d-ab9e-4777-aa6f-4ffa8c0a2de0/kube-rbac-proxy-metrics/0.log" Jul 09 18:29:03.667006 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.666972 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl"] Jul 09 18:29:03.667398 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.667318 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81caf98e-6196-4f2b-a37e-4744b0e13825" containerName="copy" Jul 09 18:29:03.667398 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.667340 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="81caf98e-6196-4f2b-a37e-4744b0e13825" containerName="copy" Jul 09 18:29:03.667398 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.667358 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81caf98e-6196-4f2b-a37e-4744b0e13825" containerName="gather" Jul 09 18:29:03.667398 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.667365 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="81caf98e-6196-4f2b-a37e-4744b0e13825" containerName="gather" Jul 09 18:29:03.667527 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.667420 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="81caf98e-6196-4f2b-a37e-4744b0e13825" containerName="copy" Jul 09 18:29:03.667527 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.667432 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="81caf98e-6196-4f2b-a37e-4744b0e13825" containerName="gather" Jul 09 18:29:03.669431 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.669412 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.672512 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.672488 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5fmck\"/\"kube-root-ca.crt\"" Jul 09 18:29:03.673214 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.673196 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5fmck\"/\"default-dockercfg-9sz9n\"" Jul 09 18:29:03.673326 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.673250 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5fmck\"/\"openshift-service-ca.crt\"" Jul 09 18:29:03.688511 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.688490 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl"] Jul 09 18:29:03.773944 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.773911 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e948a696-fdfa-466f-a751-ad1b3705f874-podres\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.774124 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.773973 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e948a696-fdfa-466f-a751-ad1b3705f874-proc\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.774124 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.774028 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e948a696-fdfa-466f-a751-ad1b3705f874-sys\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.774124 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.774066 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e948a696-fdfa-466f-a751-ad1b3705f874-lib-modules\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.774124 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.774110 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvvwd\" (UniqueName: \"kubernetes.io/projected/e948a696-fdfa-466f-a751-ad1b3705f874-kube-api-access-lvvwd\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.875110 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.875073 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e948a696-fdfa-466f-a751-ad1b3705f874-proc\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.875110 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.875112 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e948a696-fdfa-466f-a751-ad1b3705f874-sys\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.875320 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.875134 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e948a696-fdfa-466f-a751-ad1b3705f874-lib-modules\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.875320 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.875165 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvvwd\" (UniqueName: \"kubernetes.io/projected/e948a696-fdfa-466f-a751-ad1b3705f874-kube-api-access-lvvwd\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.875320 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.875221 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e948a696-fdfa-466f-a751-ad1b3705f874-sys\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.875320 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.875226 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e948a696-fdfa-466f-a751-ad1b3705f874-podres\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.875320 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.875270 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e948a696-fdfa-466f-a751-ad1b3705f874-lib-modules\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.875320 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.875297 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e948a696-fdfa-466f-a751-ad1b3705f874-podres\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.875320 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.875221 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e948a696-fdfa-466f-a751-ad1b3705f874-proc\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.884254 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.884231 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvvwd\" (UniqueName: \"kubernetes.io/projected/e948a696-fdfa-466f-a751-ad1b3705f874-kube-api-access-lvvwd\") pod \"perf-node-gather-daemonset-kjmkl\" (UID: \"e948a696-fdfa-466f-a751-ad1b3705f874\") " pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:03.982824 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.982748 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5fmck\"/\"default-dockercfg-9sz9n\"" Jul 09 18:29:03.989968 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:03.989944 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:04.137804 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:04.137772 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl"] Jul 09 18:29:04.141524 ip-10-0-137-168 kubenswrapper[2569]: W0709 18:29:04.141500 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode948a696_fdfa_466f_a751_ad1b3705f874.slice/crio-2e9ba94ad336ec0e7352cb58064587ab4ba1f983d6f5994eb8420a1a02811961 WatchSource:0}: Error finding container 2e9ba94ad336ec0e7352cb58064587ab4ba1f983d6f5994eb8420a1a02811961: Status 404 returned error can't find the container with id 2e9ba94ad336ec0e7352cb58064587ab4ba1f983d6f5994eb8420a1a02811961 Jul 09 18:29:04.818583 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:04.818545 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" event={"ID":"e948a696-fdfa-466f-a751-ad1b3705f874","Type":"ContainerStarted","Data":"4cb20f71e4a45fc31689e11412aad879e96fbb90223272c478cd97a33800fc9e"} Jul 09 18:29:04.819141 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:04.818631 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" event={"ID":"e948a696-fdfa-466f-a751-ad1b3705f874","Type":"ContainerStarted","Data":"2e9ba94ad336ec0e7352cb58064587ab4ba1f983d6f5994eb8420a1a02811961"} Jul 09 18:29:04.819141 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:04.818676 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:04.838596 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:04.838547 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" podStartSLOduration=1.838532314 podStartE2EDuration="1.838532314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:29:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:29:04.837128585 +0000 UTC m=+1501.302650997" watchObservedRunningTime="2026-07-09 18:29:04.838532314 +0000 UTC m=+1501.304054724" Jul 09 18:29:04.987010 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:04.986979 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mtt6h_4888cd81-969f-4aa9-ab7a-d9ca367fc111/dns/0.log" Jul 09 18:29:05.010976 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:05.010949 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mtt6h_4888cd81-969f-4aa9-ab7a-d9ca367fc111/kube-rbac-proxy/0.log" Jul 09 18:29:05.156015 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:05.155932 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-klp4q_0a8abea8-0e83-46f9-9fc1-f22428d89987/dns-node-resolver/0.log" Jul 09 18:29:05.744483 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:05.744454 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zs6xt_c4cada06-9af5-4483-a7a4-eca67319ff23/node-ca/0.log" Jul 09 18:29:06.936483 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:06.936451 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5pc88_f1984f61-d58e-4cc9-b883-3061273f2af5/serve-healthcheck-canary/0.log" Jul 09 18:29:07.421624 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:07.421593 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4jwbn_6d6cc2b0-cff5-4267-8cff-1d217e85385f/kube-rbac-proxy/0.log" Jul 09 18:29:07.458196 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:07.458167 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4jwbn_6d6cc2b0-cff5-4267-8cff-1d217e85385f/exporter/0.log" Jul 09 18:29:07.498589 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:07.498563 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4jwbn_6d6cc2b0-cff5-4267-8cff-1d217e85385f/extractor/0.log" Jul 09 18:29:09.811280 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:09.811248 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-748bdf6488-fk24b_128800f2-1ef6-42f5-9d53-1bc6bc35121d/manager/0.log" Jul 09 18:29:09.958817 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:09.958773 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-lsdt4_bae04c58-8e7f-4ea1-866e-15a6e8b51f9e/manager/0.log" Jul 09 18:29:10.000054 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:10.000025 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-74tq2_6244a66e-5dc5-4928-9140-4685cb2a7372/s3-init/0.log" Jul 09 18:29:10.018675 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:10.018653 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-74tq2_6244a66e-5dc5-4928-9140-4685cb2a7372/download-hf-model/0.log" Jul 09 18:29:10.834185 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:10.834156 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5fmck/perf-node-gather-daemonset-kjmkl" Jul 09 18:29:15.806238 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:15.806209 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xvhp7_d084e88e-cb6f-4344-9d93-fee760d1d9c3/kube-multus-additional-cni-plugins/0.log" Jul 09 18:29:15.831527 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:15.831503 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xvhp7_d084e88e-cb6f-4344-9d93-fee760d1d9c3/egress-router-binary-copy/0.log" Jul 09 18:29:15.856936 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:15.856907 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xvhp7_d084e88e-cb6f-4344-9d93-fee760d1d9c3/cni-plugins/0.log" Jul 09 18:29:15.883434 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:15.883411 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xvhp7_d084e88e-cb6f-4344-9d93-fee760d1d9c3/bond-cni-plugin/0.log" Jul 09 18:29:15.909144 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:15.909117 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xvhp7_d084e88e-cb6f-4344-9d93-fee760d1d9c3/routeoverride-cni/0.log" Jul 09 18:29:15.932790 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:15.932764 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xvhp7_d084e88e-cb6f-4344-9d93-fee760d1d9c3/whereabouts-cni-bincopy/0.log" Jul 09 18:29:15.959755 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:15.959724 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xvhp7_d084e88e-cb6f-4344-9d93-fee760d1d9c3/whereabouts-cni/0.log" Jul 09 18:29:16.194497 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:16.194369 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tkxzm_e9c90246-157e-41aa-9498-a676922f8f0f/kube-multus/0.log" Jul 09 18:29:16.217260 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:16.217219 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cr5qg_25941c59-99b0-40e3-98fd-687a15e8da9e/network-metrics-daemon/0.log" Jul 09 18:29:16.241530 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:16.241501 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cr5qg_25941c59-99b0-40e3-98fd-687a15e8da9e/kube-rbac-proxy/0.log" Jul 09 18:29:17.622579 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:17.622554 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjxbv_02d39fbc-ea0c-43d7-a221-f825998aa6d8/ovn-controller/0.log" Jul 09 18:29:17.645601 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:17.645579 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjxbv_02d39fbc-ea0c-43d7-a221-f825998aa6d8/ovn-acl-logging/0.log" Jul 09 18:29:17.668949 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:17.668920 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjxbv_02d39fbc-ea0c-43d7-a221-f825998aa6d8/kube-rbac-proxy-node/0.log" Jul 09 18:29:17.693518 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:17.693480 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjxbv_02d39fbc-ea0c-43d7-a221-f825998aa6d8/kube-rbac-proxy-ovn-metrics/0.log" Jul 09 18:29:17.725107 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:17.725084 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjxbv_02d39fbc-ea0c-43d7-a221-f825998aa6d8/northd/0.log" Jul 09 18:29:17.778984 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:17.778957 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjxbv_02d39fbc-ea0c-43d7-a221-f825998aa6d8/nbdb/0.log" Jul 09 18:29:17.843831 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:17.843808 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjxbv_02d39fbc-ea0c-43d7-a221-f825998aa6d8/sbdb/0.log" Jul 09 18:29:17.892832 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:17.892752 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sjxbv_02d39fbc-ea0c-43d7-a221-f825998aa6d8/ovnkube-controller/0.log" Jul 09 18:29:18.979950 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:18.979916 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-lntgg_aa5fad78-82ce-49bb-a50e-2b178586bdaf/network-check-target-container/0.log" Jul 09 18:29:20.171979 ip-10-0-137-168 kubenswrapper[2569]: I0709 18:29:20.171907 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-hspnc_3112e5e6-80ab-4355-b4cc-669776e66483/iptables-alerter/0.log"