Jul 16 09:03:13.652211 ip-10-0-134-16 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 16 09:03:13.652221 ip-10-0-134-16 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 16 09:03:13.652228 ip-10-0-134-16 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 16 09:03:13.652440 ip-10-0-134-16 systemd[1]: Failed to start Kubernetes Kubelet. Jul 16 09:03:23.809778 ip-10-0-134-16 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 16 09:03:23.809793 ip-10-0-134-16 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 683724f843b3424a85f2dc2061c223ac -- Jul 16 09:05:22.994101 ip-10-0-134-16 systemd[1]: Starting Kubernetes Kubelet... Jul 16 09:05:24.989270 ip-10-0-134-16 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 09:05:24.989270 ip-10-0-134-16 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 09:05:24.989270 ip-10-0-134-16 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 09:05:24.989270 ip-10-0-134-16 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 09:05:24.989270 ip-10-0-134-16 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 09:05:24.989270 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.958380 2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 09:05:24.989270 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962674 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 09:05:24.989270 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962685 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 09:05:24.989270 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962689 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 09:05:24.989270 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962692 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 09:05:24.989270 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962695 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 09:05:24.989270 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962698 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 09:05:24.989270 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962701 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 09:05:24.989270 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962703 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962706 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962708 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962712 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962715 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962719 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962723 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962727 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962730 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962733 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962736 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962739 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962742 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962745 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962749 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962752 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962755 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962758 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962761 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962764 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 09:05:24.990326 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962767 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962771 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962774 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962777 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962780 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962784 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962788 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962791 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962794 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962797 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962800 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962803 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962805 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962808 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962810 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962813 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962827 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962831 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962834 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 09:05:24.991592 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962836 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962839 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962842 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962845 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962848 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962850 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962853 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962856 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962860 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962863 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962866 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962869 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962871 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962874 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962877 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962880 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962883 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962885 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962889 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962891 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 09:05:24.992206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962894 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962898 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962900 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962903 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962905 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962908 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962910 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962913 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962916 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962918 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962921 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962923 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962926 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962928 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962931 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962934 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962936 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962939 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962942 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962944 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 09:05:24.993091 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962947 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962950 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962953 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962956 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962958 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962961 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962964 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962967 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962970 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962972 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962975 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962978 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.962981 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963661 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963667 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963674 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963678 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963682 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963685 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 09:05:24.994353 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963688 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963692 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963695 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963698 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963701 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963703 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963706 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963709 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963711 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963714 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963716 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963719 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963721 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963724 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963727 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963730 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963733 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963735 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963738 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963741 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 09:05:24.995204 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963744 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963746 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963749 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963752 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963754 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963757 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963759 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963761 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963764 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963767 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963770 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963772 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963775 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963777 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963780 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963783 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963785 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963788 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963790 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963793 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 09:05:24.996037 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963796 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963798 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963801 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963803 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963806 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963808 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963811 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963813 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963832 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963835 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963838 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963843 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963847 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963850 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963853 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963856 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963858 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963861 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963864 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963866 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 09:05:24.996708 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963870 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963873 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963875 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963878 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963881 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963884 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963888 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963891 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963894 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963896 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963899 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963903 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963906 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963908 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963911 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963914 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963916 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963919 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963921 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963924 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 09:05:24.997480 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963926 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963929 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963932 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963934 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963937 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963939 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963942 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963945 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963947 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963950 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963953 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963956 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.963958 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964662 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964667 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964672 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964675 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964678 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964681 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 09:05:24.998103 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964684 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964688 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964691 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964695 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964698 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964700 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964703 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964706 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964709 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964712 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964714 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964717 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964719 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964722 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964732 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964735 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964738 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964740 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964743 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964746 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 16 09:05:24.998695 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964748 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964751 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964754 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964756 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964759 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964762 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964764 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964767 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964771 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964774 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964776 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964779 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964781 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964784 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964786 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964789 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964791 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964795 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964798 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 09:05:24.999406 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964802 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964805 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964808 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964811 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964828 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964831 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964834 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964837 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964840 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964844 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964847 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964849 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964852 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964854 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964857 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964860 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964863 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964865 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964868 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 09:05:25.000000 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964871 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964874 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964879 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964882 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964884 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964887 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964889 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964892 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964894 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964897 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964900 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964902 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964905 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964907 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964910 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964912 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964915 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964917 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964921 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 09:05:25.000563 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964923 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964926 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964929 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964933 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964936 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964939 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964941 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964944 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964947 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964949 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964952 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964954 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964957 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964959 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964962 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.964966 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965036 2567 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965043 2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965048 2567 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965053 2567 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965057 2567 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 09:05:25.001420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965060 2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965064 2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965068 2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965071 2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965074 2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965077 2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965081 2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965084 2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965087 2567 flags.go:64] FLAG: --cgroup-root="" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965089 2567 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965092 2567 flags.go:64] FLAG: --client-ca-file="" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965095 2567 flags.go:64] FLAG: --cloud-provider="external" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965098 2567 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965102 2567 flags.go:64] FLAG: --cluster-domain="" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965106 2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965109 2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965112 2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965116 2567 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965119 2567 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965122 2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965125 2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965129 2567 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965132 2567 flags.go:64] FLAG: --contention-profiling="false" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965135 2567 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 09:05:25.002068 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965138 2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965141 2567 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965145 2567 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965149 2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965152 2567 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965155 2567 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965158 2567 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965161 2567 flags.go:64] FLAG: --enable-server="true" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965164 2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965168 2567 flags.go:64] FLAG: --event-burst="100" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965171 2567 flags.go:64] FLAG: --event-qps="50" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965174 2567 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965177 2567 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965180 2567 flags.go:64] FLAG: --eviction-hard="" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965184 2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965187 2567 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965190 2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965193 2567 flags.go:64] FLAG: --eviction-soft="" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965196 2567 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965199 2567 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965202 2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965205 2567 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965209 2567 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965212 2567 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965215 2567 flags.go:64] FLAG: --feature-gates="" Jul 16 09:05:25.003354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965220 2567 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965223 2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965226 2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965229 2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965232 2567 flags.go:64] FLAG: --healthz-port="10248" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965235 2567 flags.go:64] FLAG: --help="false" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965238 2567 flags.go:64] FLAG: --hostname-override="ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965241 2567 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965244 2567 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965248 2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965252 2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965255 2567 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965258 2567 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965261 2567 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965264 2567 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965267 2567 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965270 2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965273 2567 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965276 2567 flags.go:64] FLAG: --kube-reserved="" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965279 2567 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965282 2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965285 2567 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965288 2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965291 2567 flags.go:64] FLAG: --lock-file="" Jul 16 09:05:25.004388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965294 2567 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965297 2567 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965300 2567 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965305 2567 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965308 2567 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965312 2567 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965315 2567 flags.go:64] FLAG: --logging-format="text" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965317 2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965321 2567 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965324 2567 flags.go:64] FLAG: --manifest-url="" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965327 2567 flags.go:64] FLAG: --manifest-url-header="" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965335 2567 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965338 2567 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965342 2567 flags.go:64] FLAG: --max-pods="110" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965345 2567 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965348 2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965351 2567 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965355 2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965358 2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965361 2567 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965364 2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965371 2567 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965374 2567 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965377 2567 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 09:05:25.006275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965379 2567 flags.go:64] FLAG: --pod-cidr="" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965382 2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965388 2567 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965391 2567 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965394 2567 flags.go:64] FLAG: --pods-per-core="0" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965397 2567 flags.go:64] FLAG: --port="10250" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965400 2567 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965403 2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08b802495f6100d31" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965406 2567 flags.go:64] FLAG: --qos-reserved="" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965409 2567 flags.go:64] FLAG: --read-only-port="10255" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965412 2567 flags.go:64] FLAG: --register-node="true" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965415 2567 flags.go:64] FLAG: --register-with-taints="" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965419 2567 flags.go:64] FLAG: --registry-burst="10" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965423 2567 flags.go:64] FLAG: --registry-qps="5" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965426 2567 flags.go:64] FLAG: --reserved-cpus="" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965429 2567 flags.go:64] FLAG: --reserved-memory="" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965432 2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965436 2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965439 2567 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965442 2567 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965445 2567 flags.go:64] FLAG: --runonce="false" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965448 2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965451 2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965454 2567 flags.go:64] FLAG: --seccomp-default="false" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965457 2567 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 09:05:25.007109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965460 2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965464 2567 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965467 2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965470 2567 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965473 2567 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965476 2567 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965479 2567 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965482 2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965485 2567 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965488 2567 flags.go:64] FLAG: --system-cgroups="" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965491 2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965496 2567 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965499 2567 flags.go:64] FLAG: --tls-cert-file="" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965502 2567 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965505 2567 flags.go:64] FLAG: --tls-min-version="" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965508 2567 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965511 2567 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965514 2567 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965516 2567 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965519 2567 flags.go:64] FLAG: --v="2" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965525 2567 flags.go:64] FLAG: --version="false" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965528 2567 flags.go:64] FLAG: --vmodule="" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965532 2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.965535 2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965637 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 09:05:25.008409 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965641 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965644 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965647 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965650 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965653 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965656 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965659 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965662 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965667 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965670 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965673 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965676 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965679 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965681 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965684 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965687 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965689 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965692 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965694 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 09:05:25.009683 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965697 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965699 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965702 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965705 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965707 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965710 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965712 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965715 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965719 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965722 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965724 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965727 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965729 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965732 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965735 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965738 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965740 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965743 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965746 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965748 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 09:05:25.010309 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965751 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965755 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965758 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965761 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965763 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965766 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965768 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965771 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965774 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965776 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965779 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965781 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965784 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965786 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965789 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965791 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965794 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965797 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965799 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965803 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 09:05:25.010938 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965808 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965811 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965827 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965831 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965834 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965837 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965840 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965843 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965846 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965849 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965851 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965854 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965857 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965860 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965863 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965865 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965868 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965871 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965874 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 09:05:25.013179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965876 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965879 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965881 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965884 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965886 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965889 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965891 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965894 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965896 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965899 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965901 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965904 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965906 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965911 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965913 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965916 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965918 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965921 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965923 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 09:05:25.014206 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.965926 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.966982 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.976371 2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.976385 2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.977399 2567 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.977406 2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977460 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977465 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977470 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977474 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977477 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977480 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977483 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977485 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977488 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 09:05:25.015474 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977491 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977493 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977496 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977499 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977501 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977504 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977507 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977510 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977513 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977516 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977519 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977523 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977525 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977528 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977530 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977533 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977536 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977538 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977541 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977543 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 09:05:25.015960 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977546 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977548 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977551 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977554 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977557 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977560 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977563 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977565 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977568 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977570 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977575 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977578 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977581 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977584 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977587 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977590 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977593 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977596 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977599 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 09:05:25.097621 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977602 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 09:05:25.037476 ip-10-0-134-16 systemd[1]: Started Kubernetes Kubelet. Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977605 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977607 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977610 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977614 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977617 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977619 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977622 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977625 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977627 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977630 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977632 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977635 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977637 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977640 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977643 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977648 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977652 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977656 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977659 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 09:05:25.098375 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977663 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977665 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977668 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977670 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977673 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977676 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977679 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977683 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977686 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977689 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977691 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977694 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977697 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977699 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977702 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977705 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977709 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977712 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977714 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 09:05:25.099486 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977717 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977720 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977722 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977725 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977727 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977730 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977733 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977735 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977738 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977740 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977743 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977746 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.977751 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977870 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977875 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 09:05:25.100167 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977878 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977882 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977886 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977890 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977893 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977896 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977899 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977902 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977905 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977909 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977912 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977915 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977918 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977921 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977926 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977928 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977932 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977935 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977937 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 09:05:25.101690 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977940 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977943 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977946 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977949 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977952 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977955 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977957 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977960 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977963 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977965 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977969 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977972 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977975 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977978 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977980 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977982 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977985 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977988 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977990 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977993 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 09:05:25.102311 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977995 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.977998 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978001 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978003 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978006 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978009 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978012 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978015 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978018 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978021 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978023 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978026 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978028 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978031 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978034 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978036 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978039 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978041 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978043 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978046 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 09:05:25.103785 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978049 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978052 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978057 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978059 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978062 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978065 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978067 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978070 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978073 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978075 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978078 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978081 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978083 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978086 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978088 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978091 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978094 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978096 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978099 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978103 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 09:05:25.104622 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978106 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978109 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978111 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978114 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978116 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978119 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978121 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978124 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978128 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978131 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978133 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978136 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978139 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978141 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978146 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978148 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978151 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 09:05:25.106732 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:24.978154 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 09:05:25.107370 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.978158 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 09:05:25.107370 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.978253 2567 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 09:05:25.107370 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.980269 2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 09:05:25.107370 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.982391 2567 server.go:1019] "Starting client certificate rotation" Jul 16 09:05:25.107370 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.982488 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 09:05:25.107370 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.982526 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 09:05:25.107370 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.985695 2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 09:05:25.107370 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.988425 2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 09:05:25.107370 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:24.998344 2567 log.go:25] "Validated CRI v1 runtime API" Jul 16 09:05:25.107370 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.002973 2567 log.go:25] "Validated CRI v1 image API" Jul 16 09:05:25.107370 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.003626 2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 09:05:25.107370 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.006535 2567 fs.go:135] Filesystem UUIDs: map[44d5b99f-fe7a-4d90-a897-006abb408dd5:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c76601fc-0734-4965-bcfa-77b60492002b:/dev/nvme0n1p3] Jul 16 09:05:25.109131 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.006562 2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 09:05:25.109339 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.016136 2567 manager.go:216] Machine: {Timestamp:2026-07-16 09:05:25.014579495 +0000 UTC m=+0.424127764 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3049839 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e9a9a9db921c613529bedfff35139 SystemUUID:ec2e9a9a-9db9-21c6-1352-9bedfff35139 BootID:683724f8-43b3-424a-85f2-dc2061c223ac Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:19:09:f8:9a:81 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:19:09:f8:9a:81 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:e4:9f:c8:f6:07 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 09:05:25.109339 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.016255 2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 09:05:25.109339 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.016412 2567 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 09:05:25.109339 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.019402 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 09:05:25.109339 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.020075 2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 09:05:25.109339 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.020104 2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-16.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 09:05:25.109339 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.021054 2567 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.021067 2567 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.021076 2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.021900 2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.023192 2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.026805 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.026937 2567 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.026962 2567 kubelet.go:486] "Attempting to sync node with API server" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.026971 2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.026985 2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.026994 2567 kubelet.go:398] "Adding apiserver pod source" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.027001 2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.027275 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.027289 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.030037 2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.032030 2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.032056 2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.032208 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.032220 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.032226 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.032232 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.032240 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.032246 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.032872 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.032878 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.032884 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.032890 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.032900 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.032912 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.033616 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.033624 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.036852 2567 server.go:1268] "Started kubelet" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.036889 2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-16.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.036966 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.037023 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.037231 2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.039357 2567 server.go:310] "Adding debug handlers to kubelet server" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.040098 2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.040132 2567 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.041653 2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 09:05:25.112079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.044046 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ckpxj" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.045357 2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.045782 2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.045788 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.045100 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-16.ec2.internal.18c2b9d492a8138f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-16.ec2.internal,UID:ip-10-0-134-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-16.ec2.internal,},FirstTimestamp:2026-07-16 09:05:25.036831631 +0000 UTC m=+0.446379880,LastTimestamp:2026-07-16 09:05:25.036831631 +0000 UTC m=+0.446379880,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-16.ec2.internal,}" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.046124 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-16.ec2.internal\" not found" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.046490 2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.046494 2567 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.046511 2567 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.046632 2567 reconstruct.go:97] "Volume reconstruction finished" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.046640 2567 reconciler.go:29] "Reconciler: start to sync state" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.046946 2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.046958 2567 factory.go:55] Registering systemd factory Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.046968 2567 factory.go:223] Registration of the systemd container factory successfully Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.047195 2567 factory.go:153] Registering CRI-O factory Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.047204 2567 factory.go:223] Registration of the crio container factory successfully Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.047222 2567 factory.go:103] Registering Raw factory Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.047234 2567 manager.go:1196] Started watching for new ooms in manager Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.047729 2567 manager.go:349] Starting recovery of all containers Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.048616 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.048723 2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.072585 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ckpxj" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.077995 2567 manager.go:354] Recovery completed Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.082329 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.084068 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasSufficientMemory" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.084092 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.084101 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasSufficientPID" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.084616 2567 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.084623 2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.084637 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.084631 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.085845 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasSufficientMemory" Jul 16 09:05:25.113330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.085870 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.085882 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasSufficientPID" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.086377 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-16.ec2.internal.18c2b9d49579067c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-16.ec2.internal,UID:ip-10-0-134-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-16.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-16.ec2.internal,},FirstTimestamp:2026-07-16 09:05:25.08407974 +0000 UTC m=+0.493627990,LastTimestamp:2026-07-16 09:05:25.08407974 +0000 UTC m=+0.493627990,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-16.ec2.internal,}" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.089488 2567 policy_none.go:49] "None policy: Start" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.089508 2567 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.089523 2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.090578 2567 policy_none.go:47] "Start" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.135896 2567 manager.go:337] "Starting Device Plugin manager" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.135944 2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.135953 2567 server.go:87] "Starting device plugin registration server" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.136116 2567 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.136124 2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.138902 2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.138956 2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.138962 2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.139548 2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 09:05:25.190871 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.139601 2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-16.ec2.internal\" not found" Jul 16 09:05:25.264805 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.237001 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 09:05:25.264805 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.247379 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasSufficientMemory" Jul 16 09:05:25.264805 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.247405 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 09:05:25.264805 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.247415 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasSufficientPID" Jul 16 09:05:25.264805 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.247441 2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.264805 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.256579 2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.264805 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.256597 2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-16.ec2.internal\": node \"ip-10-0-134-16.ec2.internal\" not found" Jul 16 09:05:25.282990 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.277144 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-16.ec2.internal\" not found" Jul 16 09:05:25.385289 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.316321 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 09:05:25.385289 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.317746 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 09:05:25.385289 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.317786 2567 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 09:05:25.385289 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.317811 2567 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 09:05:25.385289 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.317862 2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 09:05:25.385289 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.321081 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.385498 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-16.ec2.internal\" not found" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.418653 2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-16.ec2.internal"] Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.418728 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.419712 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasSufficientMemory" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.419739 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.419749 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasSufficientPID" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.421233 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.421349 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.421379 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.422008 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasSufficientMemory" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.422032 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.422041 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasSufficientPID" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.422054 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasSufficientMemory" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.422075 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.422086 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasSufficientPID" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.423214 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.423241 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.423868 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasSufficientMemory" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.423889 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.423898 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeHasSufficientPID" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.451109 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a42d856b17d19cd675b76c4da5f6ebd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal\" (UID: \"6a42d856b17d19cd675b76c4da5f6ebd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.451131 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/23e3fcaf29c3b1a8898aed672fb43ec0-config\") pod \"kube-apiserver-proxy-ip-10-0-134-16.ec2.internal\" (UID: \"23e3fcaf29c3b1a8898aed672fb43ec0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.451147 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a42d856b17d19cd675b76c4da5f6ebd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal\" (UID: \"6a42d856b17d19cd675b76c4da5f6ebd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.465535 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-16.ec2.internal\" not found" node="ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.469865 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-16.ec2.internal\" not found" node="ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.499946 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.486579 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-16.ec2.internal\" not found" Jul 16 09:05:25.551365 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.551286 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a42d856b17d19cd675b76c4da5f6ebd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal\" (UID: \"6a42d856b17d19cd675b76c4da5f6ebd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.551365 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.551320 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/23e3fcaf29c3b1a8898aed672fb43ec0-config\") pod \"kube-apiserver-proxy-ip-10-0-134-16.ec2.internal\" (UID: \"23e3fcaf29c3b1a8898aed672fb43ec0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.551365 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.551337 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a42d856b17d19cd675b76c4da5f6ebd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal\" (UID: \"6a42d856b17d19cd675b76c4da5f6ebd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.551556 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.551384 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a42d856b17d19cd675b76c4da5f6ebd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal\" (UID: \"6a42d856b17d19cd675b76c4da5f6ebd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.551556 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.551390 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/23e3fcaf29c3b1a8898aed672fb43ec0-config\") pod \"kube-apiserver-proxy-ip-10-0-134-16.ec2.internal\" (UID: \"23e3fcaf29c3b1a8898aed672fb43ec0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.551556 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.551459 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a42d856b17d19cd675b76c4da5f6ebd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal\" (UID: \"6a42d856b17d19cd675b76c4da5f6ebd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.586779 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.586756 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-16.ec2.internal\" not found" Jul 16 09:05:25.687827 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.687787 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-16.ec2.internal\" not found" Jul 16 09:05:25.767261 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.767240 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.773701 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.773684 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-16.ec2.internal" Jul 16 09:05:25.788402 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.788384 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-16.ec2.internal\" not found" Jul 16 09:05:25.888925 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:25.888851 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-16.ec2.internal\" not found" Jul 16 09:05:25.976714 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.976695 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 09:05:25.981540 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.981524 2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 09:05:25.981650 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.981634 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 09:05:25.981693 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:25.981675 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 09:05:26.027828 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.027795 2567 apiserver.go:52] "Watching apiserver" Jul 16 09:05:26.046200 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.046175 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-16.ec2.internal" Jul 16 09:05:26.047122 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.047105 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 09:05:26.047453 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.047417 2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 09:05:26.047735 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.047718 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-lslxl","kube-system/konnectivity-agent-wvqmn","openshift-cluster-node-tuning-operator/tuned-gxzj8","openshift-multus/multus-additional-cni-plugins-sd79d","openshift-ovn-kubernetes/ovnkube-node-qqw5z","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj","openshift-dns/node-resolver-78b27","openshift-image-registry/node-ca-mprh4","openshift-multus/multus-v6mrg","openshift-multus/network-metrics-daemon-ph7bq","openshift-network-diagnostics/network-check-target-mpfsx","openshift-network-operator/iptables-alerter-8glr5"] Jul 16 09:05:26.049173 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.049158 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:26.049239 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.049221 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:26.050468 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.050456 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wvqmn" Jul 16 09:05:26.051779 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.051759 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.052777 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.052753 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n5x7r\"" Jul 16 09:05:26.052887 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.052831 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 09:05:26.052949 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.052897 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 09:05:26.052996 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.052977 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.053835 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.053809 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 09:05:26.053919 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.053898 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 09:05:26.053985 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.053968 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-64pjf\"" Jul 16 09:05:26.054106 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.054090 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.055594 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.055578 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 09:05:26.055708 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.055582 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 09:05:26.055974 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.055962 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 09:05:26.056079 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.056065 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kkt4j\"" Jul 16 09:05:26.056229 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.056209 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.056323 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.056298 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 09:05:26.056470 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.056354 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 09:05:26.056470 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.056375 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 09:05:26.056470 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.056359 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal" Jul 16 09:05:26.057930 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.057916 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 09:05:26.057930 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.057929 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 09:05:26.058054 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.057940 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 09:05:26.058054 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.057940 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 09:05:26.058054 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.057992 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-792zk\"" Jul 16 09:05:26.058054 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.057998 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 09:05:26.058326 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.058312 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 09:05:26.058391 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.058358 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6zcfz\"" Jul 16 09:05:26.058602 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.058590 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 09:05:26.058755 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.058741 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 09:05:26.058870 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.058858 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 09:05:26.060348 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.060333 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-78b27" Jul 16 09:05:26.061037 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.061022 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 09:05:26.062149 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.062135 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mprh4" Jul 16 09:05:26.062516 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.062500 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 09:05:26.062583 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.062540 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-94rz7\"" Jul 16 09:05:26.062583 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.062561 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 09:05:26.063506 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.063491 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.064394 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.064377 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 09:05:26.064483 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.064395 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6pskb\"" Jul 16 09:05:26.065167 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.065139 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 09:05:26.066429 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.065699 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 09:05:26.066429 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.065714 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:26.066429 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.066120 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 09:05:26.066429 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.066154 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:26.066429 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.066240 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fqv4x\"" Jul 16 09:05:26.068993 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.068973 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:26.069084 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.069045 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:26.069137 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.069084 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 09:05:26.069318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.069299 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-16.ec2.internal"] Jul 16 09:05:26.069398 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.069336 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal"] Jul 16 09:05:26.069398 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.069378 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8glr5" Jul 16 09:05:26.071743 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.071724 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-m275q\"" Jul 16 09:05:26.071743 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.071728 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 09:05:26.071879 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.071863 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 09:05:26.071920 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.071895 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 09:05:26.075876 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.075845 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 09:00:25 +0000 UTC" deadline="2028-02-21 08:24:02.219589356 +0000 UTC" Jul 16 09:05:26.075922 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.075876 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14039h18m36.143715669s" Jul 16 09:05:26.088963 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.088945 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-m48jd" Jul 16 09:05:26.095169 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.095143 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-m48jd" Jul 16 09:05:26.148232 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.148213 2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 09:05:26.153738 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.153720 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-registration-dir\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.153839 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.153745 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.153839 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.153762 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-run-netns\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.153839 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.153778 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-var-lib-cni-multus\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.153839 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.153802 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ebfa1019-f50b-40a9-88c7-f4842be8dbff-iptables-alerter-script\") pod \"iptables-alerter-8glr5\" (UID: \"ebfa1019-f50b-40a9-88c7-f4842be8dbff\") " pod="openshift-network-operator/iptables-alerter-8glr5" Jul 16 09:05:26.154026 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.153872 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ebfa1019-f50b-40a9-88c7-f4842be8dbff-host-slash\") pod \"iptables-alerter-8glr5\" (UID: \"ebfa1019-f50b-40a9-88c7-f4842be8dbff\") " pod="openshift-network-operator/iptables-alerter-8glr5" Jul 16 09:05:26.154026 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.153904 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-sysctl-d\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.154026 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.153921 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-var-lib-kubelet\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.154026 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.153953 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-kubelet\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.154026 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.153983 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-slash\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.154026 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154012 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-cni-bin\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.154239 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154035 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-os-release\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.154239 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154058 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-var-lib-cni-bin\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.154239 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154088 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-var-lib-kubelet\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.154239 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154114 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnkwf\" (UniqueName: \"kubernetes.io/projected/21c0ee12-ae01-43a6-83b1-af6b69997b76-kube-api-access-hnkwf\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.154239 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154135 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-host\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.154239 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154154 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21c0ee12-ae01-43a6-83b1-af6b69997b76-cnibin\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.154239 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154170 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/21c0ee12-ae01-43a6-83b1-af6b69997b76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.154239 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154185 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6c4d81b5-6013-424d-be5e-ec4877e5d16c-multus-daemon-config\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.154239 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154199 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:26.154239 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154226 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj6b5\" (UniqueName: \"kubernetes.io/projected/3f14599c-3b35-405b-a145-1a861e050d1c-kube-api-access-jj6b5\") pod \"node-ca-mprh4\" (UID: \"3f14599c-3b35-405b-a145-1a861e050d1c\") " pod="openshift-image-registry/node-ca-mprh4" Jul 16 09:05:26.154510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154252 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfkq9\" (UniqueName: \"kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9\") pod \"network-check-target-mpfsx\" (UID: \"47a724a3-7c9b-4b6c-bd58-293cb309739f\") " pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:26.154510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154279 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-lib-modules\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.154510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154301 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21c0ee12-ae01-43a6-83b1-af6b69997b76-system-cni-dir\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.154510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154317 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-device-dir\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.154510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154331 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-sys-fs\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.154510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154345 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/850750a5-782b-490b-b29a-ae2e05d1aff8-hosts-file\") pod \"node-resolver-78b27\" (UID: \"850750a5-782b-490b-b29a-ae2e05d1aff8\") " pod="openshift-dns/node-resolver-78b27" Jul 16 09:05:26.154510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154359 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l56r4\" (UniqueName: \"kubernetes.io/projected/850750a5-782b-490b-b29a-ae2e05d1aff8-kube-api-access-l56r4\") pod \"node-resolver-78b27\" (UID: \"850750a5-782b-490b-b29a-ae2e05d1aff8\") " pod="openshift-dns/node-resolver-78b27" Jul 16 09:05:26.154510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154380 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-run-k8s-cni-cncf-io\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.154510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154394 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-log-socket\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.154510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154407 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.154510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154434 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8zsj\" (UniqueName: \"kubernetes.io/projected/2615540b-77e1-4634-8099-34643fa92d7a-kube-api-access-z8zsj\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.154510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154455 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9eb0a970-f928-4d04-ab26-a2e41414db37-agent-certs\") pod \"konnectivity-agent-wvqmn\" (UID: \"9eb0a970-f928-4d04-ab26-a2e41414db37\") " pod="kube-system/konnectivity-agent-wvqmn" Jul 16 09:05:26.154510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154471 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxvp4\" (UniqueName: \"kubernetes.io/projected/ebfa1019-f50b-40a9-88c7-f4842be8dbff-kube-api-access-vxvp4\") pod \"iptables-alerter-8glr5\" (UID: \"ebfa1019-f50b-40a9-88c7-f4842be8dbff\") " pod="openshift-network-operator/iptables-alerter-8glr5" Jul 16 09:05:26.154913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154486 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-kubernetes\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.154913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154499 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-run-systemd\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.154913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154520 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-node-log\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.154913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154540 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-multus-cni-dir\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.154913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154554 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e84aa8e1-6582-487b-8261-61d858852378-kubelet-config\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:26.154913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154567 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-modprobe-d\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.154913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154580 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b51be5d2-0338-4152-9637-445cd871a7a9-tmp\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.154913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154594 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:26.154913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154608 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38965c3a-8c17-4db6-af25-f28e481859dc-env-overrides\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.154913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154622 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38965c3a-8c17-4db6-af25-f28e481859dc-ovn-node-metrics-cert\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.154913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154635 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f14599c-3b35-405b-a145-1a861e050d1c-host\") pod \"node-ca-mprh4\" (UID: \"3f14599c-3b35-405b-a145-1a861e050d1c\") " pod="openshift-image-registry/node-ca-mprh4" Jul 16 09:05:26.154913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154647 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b51be5d2-0338-4152-9637-445cd871a7a9-etc-tuned\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.154913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154661 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/850750a5-782b-490b-b29a-ae2e05d1aff8-tmp-dir\") pod \"node-resolver-78b27\" (UID: \"850750a5-782b-490b-b29a-ae2e05d1aff8\") " pod="openshift-dns/node-resolver-78b27" Jul 16 09:05:26.154913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154675 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-multus-socket-dir-parent\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.155436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154691 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-multus-conf-dir\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.155436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154713 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-etc-kubernetes\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.155436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154726 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr864\" (UniqueName: \"kubernetes.io/projected/6c4d81b5-6013-424d-be5e-ec4877e5d16c-kube-api-access-zr864\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.155436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154740 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f14599c-3b35-405b-a145-1a861e050d1c-serviceca\") pod \"node-ca-mprh4\" (UID: \"3f14599c-3b35-405b-a145-1a861e050d1c\") " pod="openshift-image-registry/node-ca-mprh4" Jul 16 09:05:26.155436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154754 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/21c0ee12-ae01-43a6-83b1-af6b69997b76-os-release\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.155436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154768 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-etc-openvswitch\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.155436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154781 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-run-openvswitch\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.155436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154794 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38965c3a-8c17-4db6-af25-f28e481859dc-ovnkube-script-lib\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.155436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154813 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e84aa8e1-6582-487b-8261-61d858852378-dbus\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:26.155436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154846 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8bmr\" (UniqueName: \"kubernetes.io/projected/b51be5d2-0338-4152-9637-445cd871a7a9-kube-api-access-n8bmr\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.155436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154869 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/21c0ee12-ae01-43a6-83b1-af6b69997b76-cni-binary-copy\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.155436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154895 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21c0ee12-ae01-43a6-83b1-af6b69997b76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.155436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154909 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.155436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154928 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-var-lib-openvswitch\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.155810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154942 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-cnibin\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.155810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154957 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6c4d81b5-6013-424d-be5e-ec4877e5d16c-cni-binary-copy\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.155810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154972 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6rdj\" (UniqueName: \"kubernetes.io/projected/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-kube-api-access-g6rdj\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:26.155810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.154986 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-systemd\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.155810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155006 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-systemd-units\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.155810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155027 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-cni-netd\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.155810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155042 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.155810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155056 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-sys\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.155810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155069 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38965c3a-8c17-4db6-af25-f28e481859dc-ovnkube-config\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.155810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155115 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-system-cni-dir\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.155810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155152 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-hostroot\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.155810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155177 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-run-multus-certs\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.155810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155203 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-sysconfig\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.155810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155225 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-sysctl-conf\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.156235 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155248 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-run\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.156235 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155270 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-run-netns\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.156235 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155294 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-run-ovn\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.156235 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155314 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-socket-dir\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.156235 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155338 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9eb0a970-f928-4d04-ab26-a2e41414db37-konnectivity-ca\") pod \"konnectivity-agent-wvqmn\" (UID: \"9eb0a970-f928-4d04-ab26-a2e41414db37\") " pod="kube-system/konnectivity-agent-wvqmn" Jul 16 09:05:26.156235 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155362 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/21c0ee12-ae01-43a6-83b1-af6b69997b76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.156235 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.155386 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fnns\" (UniqueName: \"kubernetes.io/projected/38965c3a-8c17-4db6-af25-f28e481859dc-kube-api-access-4fnns\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.255876 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.255846 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-sysconfig\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.255876 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.255875 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-sysctl-conf\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.256094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.255892 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-run\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.256094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.255907 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-run-netns\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.256094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.255927 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-run-ovn\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.256094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.255978 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-run-ovn\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.256094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.255980 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-sysconfig\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.256094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256010 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-socket-dir\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.256094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256028 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9eb0a970-f928-4d04-ab26-a2e41414db37-konnectivity-ca\") pod \"konnectivity-agent-wvqmn\" (UID: \"9eb0a970-f928-4d04-ab26-a2e41414db37\") " pod="kube-system/konnectivity-agent-wvqmn" Jul 16 09:05:26.256094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256049 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/21c0ee12-ae01-43a6-83b1-af6b69997b76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.256094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256053 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-run\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.256094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256047 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-sysctl-conf\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.256094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256074 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fnns\" (UniqueName: \"kubernetes.io/projected/38965c3a-8c17-4db6-af25-f28e481859dc-kube-api-access-4fnns\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.256094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256058 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-run-netns\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.256094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256097 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-registration-dir\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.256704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256147 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-registration-dir\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.256704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256179 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-socket-dir\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.256704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256200 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.256704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256235 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-run-netns\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.256704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256243 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.256704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256259 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-var-lib-cni-multus\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.256704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256298 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ebfa1019-f50b-40a9-88c7-f4842be8dbff-iptables-alerter-script\") pod \"iptables-alerter-8glr5\" (UID: \"ebfa1019-f50b-40a9-88c7-f4842be8dbff\") " pod="openshift-network-operator/iptables-alerter-8glr5" Jul 16 09:05:26.256704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256305 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-run-netns\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.256704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256321 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ebfa1019-f50b-40a9-88c7-f4842be8dbff-host-slash\") pod \"iptables-alerter-8glr5\" (UID: \"ebfa1019-f50b-40a9-88c7-f4842be8dbff\") " pod="openshift-network-operator/iptables-alerter-8glr5" Jul 16 09:05:26.256704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256343 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-sysctl-d\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.256704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256366 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-var-lib-kubelet\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.256704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256395 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-kubelet\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.256704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256417 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-slash\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.256704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256436 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-var-lib-kubelet\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256439 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-cni-bin\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256475 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-os-release\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256500 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-var-lib-cni-bin\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256502 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-sysctl-d\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256518 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-slash\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256525 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-var-lib-kubelet\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256478 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-cni-bin\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256549 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ebfa1019-f50b-40a9-88c7-f4842be8dbff-host-slash\") pod \"iptables-alerter-8glr5\" (UID: \"ebfa1019-f50b-40a9-88c7-f4842be8dbff\") " pod="openshift-network-operator/iptables-alerter-8glr5" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256559 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnkwf\" (UniqueName: \"kubernetes.io/projected/21c0ee12-ae01-43a6-83b1-af6b69997b76-kube-api-access-hnkwf\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256572 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-var-lib-cni-bin\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256583 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-host\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256586 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-kubelet\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256588 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-var-lib-kubelet\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256637 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-host\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.257362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256643 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-os-release\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.258117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256634 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-var-lib-cni-multus\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.258117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256653 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/21c0ee12-ae01-43a6-83b1-af6b69997b76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.258117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256694 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21c0ee12-ae01-43a6-83b1-af6b69997b76-cnibin\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.258117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256657 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9eb0a970-f928-4d04-ab26-a2e41414db37-konnectivity-ca\") pod \"konnectivity-agent-wvqmn\" (UID: \"9eb0a970-f928-4d04-ab26-a2e41414db37\") " pod="kube-system/konnectivity-agent-wvqmn" Jul 16 09:05:26.258117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256660 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21c0ee12-ae01-43a6-83b1-af6b69997b76-cnibin\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.258117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256734 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/21c0ee12-ae01-43a6-83b1-af6b69997b76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.258117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256755 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6c4d81b5-6013-424d-be5e-ec4877e5d16c-multus-daemon-config\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.258117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256761 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ebfa1019-f50b-40a9-88c7-f4842be8dbff-iptables-alerter-script\") pod \"iptables-alerter-8glr5\" (UID: \"ebfa1019-f50b-40a9-88c7-f4842be8dbff\") " pod="openshift-network-operator/iptables-alerter-8glr5" Jul 16 09:05:26.258117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256770 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:26.258117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256787 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj6b5\" (UniqueName: \"kubernetes.io/projected/3f14599c-3b35-405b-a145-1a861e050d1c-kube-api-access-jj6b5\") pod \"node-ca-mprh4\" (UID: \"3f14599c-3b35-405b-a145-1a861e050d1c\") " pod="openshift-image-registry/node-ca-mprh4" Jul 16 09:05:26.258117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256808 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfkq9\" (UniqueName: \"kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9\") pod \"network-check-target-mpfsx\" (UID: \"47a724a3-7c9b-4b6c-bd58-293cb309739f\") " pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:26.258117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256845 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-lib-modules\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.258117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256871 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21c0ee12-ae01-43a6-83b1-af6b69997b76-system-cni-dir\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.258117 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.256893 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 09:05:26.258692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256894 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-device-dir\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.258692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256936 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-device-dir\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.258692 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.256951 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret podName:e84aa8e1-6582-487b-8261-61d858852378 nodeName:}" failed. No retries permitted until 2026-07-16 09:05:26.756932334 +0000 UTC m=+2.166480585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret") pod "global-pull-secret-syncer-lslxl" (UID: "e84aa8e1-6582-487b-8261-61d858852378") : object "kube-system"/"original-pull-secret" not registered Jul 16 09:05:26.258692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256979 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21c0ee12-ae01-43a6-83b1-af6b69997b76-system-cni-dir\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.258692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.256978 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-sys-fs\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.258692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257017 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-lib-modules\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.258692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257030 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-sys-fs\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.258692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257019 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/850750a5-782b-490b-b29a-ae2e05d1aff8-hosts-file\") pod \"node-resolver-78b27\" (UID: \"850750a5-782b-490b-b29a-ae2e05d1aff8\") " pod="openshift-dns/node-resolver-78b27" Jul 16 09:05:26.258692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257058 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l56r4\" (UniqueName: \"kubernetes.io/projected/850750a5-782b-490b-b29a-ae2e05d1aff8-kube-api-access-l56r4\") pod \"node-resolver-78b27\" (UID: \"850750a5-782b-490b-b29a-ae2e05d1aff8\") " pod="openshift-dns/node-resolver-78b27" Jul 16 09:05:26.258692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257075 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-run-k8s-cni-cncf-io\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.258692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257057 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/850750a5-782b-490b-b29a-ae2e05d1aff8-hosts-file\") pod \"node-resolver-78b27\" (UID: \"850750a5-782b-490b-b29a-ae2e05d1aff8\") " pod="openshift-dns/node-resolver-78b27" Jul 16 09:05:26.258692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257094 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-log-socket\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.258692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257120 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.258692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257121 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-run-k8s-cni-cncf-io\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.259318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257124 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-log-socket\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.259318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257160 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8zsj\" (UniqueName: \"kubernetes.io/projected/2615540b-77e1-4634-8099-34643fa92d7a-kube-api-access-z8zsj\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.259318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257172 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.259318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257202 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9eb0a970-f928-4d04-ab26-a2e41414db37-agent-certs\") pod \"konnectivity-agent-wvqmn\" (UID: \"9eb0a970-f928-4d04-ab26-a2e41414db37\") " pod="kube-system/konnectivity-agent-wvqmn" Jul 16 09:05:26.259318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257211 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/21c0ee12-ae01-43a6-83b1-af6b69997b76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.259318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257294 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxvp4\" (UniqueName: \"kubernetes.io/projected/ebfa1019-f50b-40a9-88c7-f4842be8dbff-kube-api-access-vxvp4\") pod \"iptables-alerter-8glr5\" (UID: \"ebfa1019-f50b-40a9-88c7-f4842be8dbff\") " pod="openshift-network-operator/iptables-alerter-8glr5" Jul 16 09:05:26.259318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257352 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-kubernetes\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.259318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257377 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-run-systemd\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.259318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257401 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-node-log\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.259318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257420 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6c4d81b5-6013-424d-be5e-ec4877e5d16c-multus-daemon-config\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.259318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257426 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-multus-cni-dir\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.259318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257452 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-run-systemd\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.259318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257455 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-kubernetes\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.259318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257474 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-node-log\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257476 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e84aa8e1-6582-487b-8261-61d858852378-kubelet-config\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257508 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-modprobe-d\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257514 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e84aa8e1-6582-487b-8261-61d858852378-kubelet-config\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257517 2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257531 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b51be5d2-0338-4152-9637-445cd871a7a9-tmp\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257556 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257581 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38965c3a-8c17-4db6-af25-f28e481859dc-env-overrides\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257594 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-multus-cni-dir\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257603 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-modprobe-d\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257605 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38965c3a-8c17-4db6-af25-f28e481859dc-ovn-node-metrics-cert\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257654 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f14599c-3b35-405b-a145-1a861e050d1c-host\") pod \"node-ca-mprh4\" (UID: \"3f14599c-3b35-405b-a145-1a861e050d1c\") " pod="openshift-image-registry/node-ca-mprh4" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257669 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b51be5d2-0338-4152-9637-445cd871a7a9-etc-tuned\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.257655 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257692 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/850750a5-782b-490b-b29a-ae2e05d1aff8-tmp-dir\") pod \"node-resolver-78b27\" (UID: \"850750a5-782b-490b-b29a-ae2e05d1aff8\") " pod="openshift-dns/node-resolver-78b27" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257710 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f14599c-3b35-405b-a145-1a861e050d1c-host\") pod \"node-ca-mprh4\" (UID: \"3f14599c-3b35-405b-a145-1a861e050d1c\") " pod="openshift-image-registry/node-ca-mprh4" Jul 16 09:05:26.260040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257716 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-multus-socket-dir-parent\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.260780 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.257747 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs podName:9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd nodeName:}" failed. No retries permitted until 2026-07-16 09:05:26.757728583 +0000 UTC m=+2.167276835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs") pod "network-metrics-daemon-ph7bq" (UID: "9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 09:05:26.260780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257768 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-multus-socket-dir-parent\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.260780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257793 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-multus-conf-dir\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.260780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257835 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-etc-kubernetes\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.260780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257863 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr864\" (UniqueName: \"kubernetes.io/projected/6c4d81b5-6013-424d-be5e-ec4877e5d16c-kube-api-access-zr864\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.260780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257888 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f14599c-3b35-405b-a145-1a861e050d1c-serviceca\") pod \"node-ca-mprh4\" (UID: \"3f14599c-3b35-405b-a145-1a861e050d1c\") " pod="openshift-image-registry/node-ca-mprh4" Jul 16 09:05:26.260780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257894 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-multus-conf-dir\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.260780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257912 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/21c0ee12-ae01-43a6-83b1-af6b69997b76-os-release\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.260780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257956 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-etc-openvswitch\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.260780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257962 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/21c0ee12-ae01-43a6-83b1-af6b69997b76-os-release\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.260780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.257993 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-run-openvswitch\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.260780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258018 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/850750a5-782b-490b-b29a-ae2e05d1aff8-tmp-dir\") pod \"node-resolver-78b27\" (UID: \"850750a5-782b-490b-b29a-ae2e05d1aff8\") " pod="openshift-dns/node-resolver-78b27" Jul 16 09:05:26.260780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258027 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/38965c3a-8c17-4db6-af25-f28e481859dc-env-overrides\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.260780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258018 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-etc-openvswitch\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.261230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258019 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38965c3a-8c17-4db6-af25-f28e481859dc-ovnkube-script-lib\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.261230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258102 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-etc-kubernetes\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.261230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258173 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-run-openvswitch\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.261230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258224 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e84aa8e1-6582-487b-8261-61d858852378-dbus\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:26.261230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258268 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8bmr\" (UniqueName: \"kubernetes.io/projected/b51be5d2-0338-4152-9637-445cd871a7a9-kube-api-access-n8bmr\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.261230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258302 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/21c0ee12-ae01-43a6-83b1-af6b69997b76-cni-binary-copy\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.261230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258374 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21c0ee12-ae01-43a6-83b1-af6b69997b76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.261230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258399 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.261230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258441 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-var-lib-openvswitch\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.261230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258466 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-cnibin\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.261230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258505 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6c4d81b5-6013-424d-be5e-ec4877e5d16c-cni-binary-copy\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.261230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258532 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6rdj\" (UniqueName: \"kubernetes.io/projected/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-kube-api-access-g6rdj\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:26.261230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258550 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/38965c3a-8c17-4db6-af25-f28e481859dc-ovnkube-script-lib\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.261230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258556 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-systemd\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258582 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f14599c-3b35-405b-a145-1a861e050d1c-serviceca\") pod \"node-ca-mprh4\" (UID: \"3f14599c-3b35-405b-a145-1a861e050d1c\") " pod="openshift-image-registry/node-ca-mprh4" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258597 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-systemd-units\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258620 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-cni-netd\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258660 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258671 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2615540b-77e1-4634-8099-34643fa92d7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258683 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-sys\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258706 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38965c3a-8c17-4db6-af25-f28e481859dc-ovnkube-config\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258803 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e84aa8e1-6582-487b-8261-61d858852378-dbus\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258807 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-etc-systemd\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258877 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-cni-netd\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258886 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-var-lib-openvswitch\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258923 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-systemd-units\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258929 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38965c3a-8c17-4db6-af25-f28e481859dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.258973 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-cnibin\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.261601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.259064 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21c0ee12-ae01-43a6-83b1-af6b69997b76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.262078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.259118 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b51be5d2-0338-4152-9637-445cd871a7a9-sys\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.262078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.259122 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/21c0ee12-ae01-43a6-83b1-af6b69997b76-cni-binary-copy\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.262078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.259358 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-system-cni-dir\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.262078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.259394 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-hostroot\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.262078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.259417 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6c4d81b5-6013-424d-be5e-ec4877e5d16c-cni-binary-copy\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.262078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.259440 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-run-multus-certs\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.262078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.259496 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-host-run-multus-certs\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.262078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.259516 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-hostroot\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.262078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.259540 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c4d81b5-6013-424d-be5e-ec4877e5d16c-system-cni-dir\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.262078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.259636 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/38965c3a-8c17-4db6-af25-f28e481859dc-ovnkube-config\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.262078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.260983 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b51be5d2-0338-4152-9637-445cd871a7a9-etc-tuned\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.262078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.261137 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/38965c3a-8c17-4db6-af25-f28e481859dc-ovn-node-metrics-cert\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.262078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.261175 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9eb0a970-f928-4d04-ab26-a2e41414db37-agent-certs\") pod \"konnectivity-agent-wvqmn\" (UID: \"9eb0a970-f928-4d04-ab26-a2e41414db37\") " pod="kube-system/konnectivity-agent-wvqmn" Jul 16 09:05:26.262078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.261302 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b51be5d2-0338-4152-9637-445cd871a7a9-tmp\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.264615 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.264592 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnkwf\" (UniqueName: \"kubernetes.io/projected/21c0ee12-ae01-43a6-83b1-af6b69997b76-kube-api-access-hnkwf\") pod \"multus-additional-cni-plugins-sd79d\" (UID: \"21c0ee12-ae01-43a6-83b1-af6b69997b76\") " pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.264687 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.264623 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fnns\" (UniqueName: \"kubernetes.io/projected/38965c3a-8c17-4db6-af25-f28e481859dc-kube-api-access-4fnns\") pod \"ovnkube-node-qqw5z\" (UID: \"38965c3a-8c17-4db6-af25-f28e481859dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.266309 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.266293 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 09:05:26.266354 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.266314 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 09:05:26.266354 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.266333 2567 projected.go:196] Error preparing data for projected volume kube-api-access-pfkq9 for pod openshift-network-diagnostics/network-check-target-mpfsx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 09:05:26.266420 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.266392 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9 podName:47a724a3-7c9b-4b6c-bd58-293cb309739f nodeName:}" failed. No retries permitted until 2026-07-16 09:05:26.766378542 +0000 UTC m=+2.175926812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pfkq9" (UniqueName: "kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9") pod "network-check-target-mpfsx" (UID: "47a724a3-7c9b-4b6c-bd58-293cb309739f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 09:05:26.268387 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.268369 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8bmr\" (UniqueName: \"kubernetes.io/projected/b51be5d2-0338-4152-9637-445cd871a7a9-kube-api-access-n8bmr\") pod \"tuned-gxzj8\" (UID: \"b51be5d2-0338-4152-9637-445cd871a7a9\") " pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.271413 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.271386 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6rdj\" (UniqueName: \"kubernetes.io/projected/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-kube-api-access-g6rdj\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:26.273550 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.273523 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj6b5\" (UniqueName: \"kubernetes.io/projected/3f14599c-3b35-405b-a145-1a861e050d1c-kube-api-access-jj6b5\") pod \"node-ca-mprh4\" (UID: \"3f14599c-3b35-405b-a145-1a861e050d1c\") " pod="openshift-image-registry/node-ca-mprh4" Jul 16 09:05:26.274146 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.274127 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l56r4\" (UniqueName: \"kubernetes.io/projected/850750a5-782b-490b-b29a-ae2e05d1aff8-kube-api-access-l56r4\") pod \"node-resolver-78b27\" (UID: \"850750a5-782b-490b-b29a-ae2e05d1aff8\") " pod="openshift-dns/node-resolver-78b27" Jul 16 09:05:26.274203 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.274189 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8zsj\" (UniqueName: \"kubernetes.io/projected/2615540b-77e1-4634-8099-34643fa92d7a-kube-api-access-z8zsj\") pod \"aws-ebs-csi-driver-node-9hhcj\" (UID: \"2615540b-77e1-4634-8099-34643fa92d7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.274483 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.274464 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxvp4\" (UniqueName: \"kubernetes.io/projected/ebfa1019-f50b-40a9-88c7-f4842be8dbff-kube-api-access-vxvp4\") pod \"iptables-alerter-8glr5\" (UID: \"ebfa1019-f50b-40a9-88c7-f4842be8dbff\") " pod="openshift-network-operator/iptables-alerter-8glr5" Jul 16 09:05:26.274799 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.274785 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr864\" (UniqueName: \"kubernetes.io/projected/6c4d81b5-6013-424d-be5e-ec4877e5d16c-kube-api-access-zr864\") pod \"multus-v6mrg\" (UID: \"6c4d81b5-6013-424d-be5e-ec4877e5d16c\") " pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.366896 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.366874 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 09:05:26.369325 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.369300 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wvqmn" Jul 16 09:05:26.370621 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.370605 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 09:05:26.396625 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.396567 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" Jul 16 09:05:26.414095 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.414076 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sd79d" Jul 16 09:05:26.428694 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.428676 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:26.442475 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.442453 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" Jul 16 09:05:26.446466 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.446448 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-78b27" Jul 16 09:05:26.461125 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.461109 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mprh4" Jul 16 09:05:26.476893 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.476873 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v6mrg" Jul 16 09:05:26.480474 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.480460 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8glr5" Jul 16 09:05:26.764331 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.764305 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:26.764471 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.764356 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:26.764471 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.764453 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 09:05:26.764471 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.764454 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 09:05:26.764575 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.764509 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret podName:e84aa8e1-6582-487b-8261-61d858852378 nodeName:}" failed. No retries permitted until 2026-07-16 09:05:27.764492248 +0000 UTC m=+3.174040487 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret") pod "global-pull-secret-syncer-lslxl" (UID: "e84aa8e1-6582-487b-8261-61d858852378") : object "kube-system"/"original-pull-secret" not registered Jul 16 09:05:26.764575 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.764523 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs podName:9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd nodeName:}" failed. No retries permitted until 2026-07-16 09:05:27.764517055 +0000 UTC m=+3.174065291 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs") pod "network-metrics-daemon-ph7bq" (UID: "9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 09:05:26.865215 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.865182 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfkq9\" (UniqueName: \"kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9\") pod \"network-check-target-mpfsx\" (UID: \"47a724a3-7c9b-4b6c-bd58-293cb309739f\") " pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:26.865481 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.865322 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 09:05:26.865481 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.865338 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 09:05:26.865481 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.865347 2567 projected.go:196] Error preparing data for projected volume kube-api-access-pfkq9 for pod openshift-network-diagnostics/network-check-target-mpfsx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 09:05:26.865481 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:26.865399 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9 podName:47a724a3-7c9b-4b6c-bd58-293cb309739f nodeName:}" failed. No retries permitted until 2026-07-16 09:05:27.865383916 +0000 UTC m=+3.274932154 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfkq9" (UniqueName: "kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9") pod "network-check-target-mpfsx" (UID: "47a724a3-7c9b-4b6c-bd58-293cb309739f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 09:05:26.893282 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:26.893238 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2615540b_77e1_4634_8099_34643fa92d7a.slice/crio-6a085c46474dfde358640f23ad26dc6d844364253354ffe5446365fc04c06184 WatchSource:0}: Error finding container 6a085c46474dfde358640f23ad26dc6d844364253354ffe5446365fc04c06184: Status 404 returned error can't find the container with id 6a085c46474dfde358640f23ad26dc6d844364253354ffe5446365fc04c06184 Jul 16 09:05:26.893652 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:26.893626 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a42d856b17d19cd675b76c4da5f6ebd.slice/crio-b89df6af1906d86203f847f48255c3b1a77f61d4d37519682a8f756d9ec4e10d WatchSource:0}: Error finding container b89df6af1906d86203f847f48255c3b1a77f61d4d37519682a8f756d9ec4e10d: Status 404 returned error can't find the container with id b89df6af1906d86203f847f48255c3b1a77f61d4d37519682a8f756d9ec4e10d Jul 16 09:05:26.894503 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:26.894478 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c4d81b5_6013_424d_be5e_ec4877e5d16c.slice/crio-37dc55394a60b5c62d4b91e0925acf448644f729fc3a9764141a7f073f7c32df WatchSource:0}: Error finding container 37dc55394a60b5c62d4b91e0925acf448644f729fc3a9764141a7f073f7c32df: Status 404 returned error can't find the container with id 37dc55394a60b5c62d4b91e0925acf448644f729fc3a9764141a7f073f7c32df Jul 16 09:05:26.898492 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:26.898452 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21c0ee12_ae01_43a6_83b1_af6b69997b76.slice/crio-5d2584289736861a4b6941d7366eaa6e84c8c681b4ca06ffa88a9c57c70b0fc9 WatchSource:0}: Error finding container 5d2584289736861a4b6941d7366eaa6e84c8c681b4ca06ffa88a9c57c70b0fc9: Status 404 returned error can't find the container with id 5d2584289736861a4b6941d7366eaa6e84c8c681b4ca06ffa88a9c57c70b0fc9 Jul 16 09:05:26.898954 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:26.898937 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 09:05:26.916381 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:26.916364 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod850750a5_782b_490b_b29a_ae2e05d1aff8.slice/crio-fada4d68935518a51c0b376f5acac2355698c385f443d1a3ca9f7969fb7b09b6 WatchSource:0}: Error finding container fada4d68935518a51c0b376f5acac2355698c385f443d1a3ca9f7969fb7b09b6: Status 404 returned error can't find the container with id fada4d68935518a51c0b376f5acac2355698c385f443d1a3ca9f7969fb7b09b6 Jul 16 09:05:26.938908 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:26.938891 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb0a970_f928_4d04_ab26_a2e41414db37.slice/crio-5c6cb2519acbcacf717f5e405f6117572b196f8310bb871ba58f966641c8438f WatchSource:0}: Error finding container 5c6cb2519acbcacf717f5e405f6117572b196f8310bb871ba58f966641c8438f: Status 404 returned error can't find the container with id 5c6cb2519acbcacf717f5e405f6117572b196f8310bb871ba58f966641c8438f Jul 16 09:05:26.962045 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:26.962025 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23e3fcaf29c3b1a8898aed672fb43ec0.slice/crio-2f11991b9c058b2dd52bc6d93c8dbc69cb2bc0c464b6b8eac5931ab0ff46b0e7 WatchSource:0}: Error finding container 2f11991b9c058b2dd52bc6d93c8dbc69cb2bc0c464b6b8eac5931ab0ff46b0e7: Status 404 returned error can't find the container with id 2f11991b9c058b2dd52bc6d93c8dbc69cb2bc0c464b6b8eac5931ab0ff46b0e7 Jul 16 09:05:26.962503 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:26.962489 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f14599c_3b35_405b_a145_1a861e050d1c.slice/crio-e80ba98bb7b0bab9e2bcd256d043da44767247b386272d84fc76327c7b24ecd0 WatchSource:0}: Error finding container e80ba98bb7b0bab9e2bcd256d043da44767247b386272d84fc76327c7b24ecd0: Status 404 returned error can't find the container with id e80ba98bb7b0bab9e2bcd256d043da44767247b386272d84fc76327c7b24ecd0 Jul 16 09:05:27.073471 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:27.073449 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38965c3a_8c17_4db6_af25_f28e481859dc.slice/crio-8b2c5f56e12b78fde4a550406ae5cdd57ea57f24d7359ef33aead2a5e0ee664e WatchSource:0}: Error finding container 8b2c5f56e12b78fde4a550406ae5cdd57ea57f24d7359ef33aead2a5e0ee664e: Status 404 returned error can't find the container with id 8b2c5f56e12b78fde4a550406ae5cdd57ea57f24d7359ef33aead2a5e0ee664e Jul 16 09:05:27.097606 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.097577 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 09:00:26 +0000 UTC" deadline="2028-01-31 09:19:03.607681789 +0000 UTC" Jul 16 09:05:27.097606 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.097603 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13536h13m36.510082191s" Jul 16 09:05:27.112092 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:27.112070 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb51be5d2_0338_4152_9637_445cd871a7a9.slice/crio-51072fa8bdc034122d072cd278844fd0deb9d4a33008da4fb16a6a65e123b697 WatchSource:0}: Error finding container 51072fa8bdc034122d072cd278844fd0deb9d4a33008da4fb16a6a65e123b697: Status 404 returned error can't find the container with id 51072fa8bdc034122d072cd278844fd0deb9d4a33008da4fb16a6a65e123b697 Jul 16 09:05:27.318112 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.318080 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:27.318259 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:27.318207 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:27.323476 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.323390 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" event={"ID":"b51be5d2-0338-4152-9637-445cd871a7a9","Type":"ContainerStarted","Data":"51072fa8bdc034122d072cd278844fd0deb9d4a33008da4fb16a6a65e123b697"} Jul 16 09:05:27.324346 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.324321 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mprh4" event={"ID":"3f14599c-3b35-405b-a145-1a861e050d1c","Type":"ContainerStarted","Data":"e80ba98bb7b0bab9e2bcd256d043da44767247b386272d84fc76327c7b24ecd0"} Jul 16 09:05:27.325270 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.325242 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-16.ec2.internal" event={"ID":"23e3fcaf29c3b1a8898aed672fb43ec0","Type":"ContainerStarted","Data":"2f11991b9c058b2dd52bc6d93c8dbc69cb2bc0c464b6b8eac5931ab0ff46b0e7"} Jul 16 09:05:27.326224 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.326203 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd79d" event={"ID":"21c0ee12-ae01-43a6-83b1-af6b69997b76","Type":"ContainerStarted","Data":"5d2584289736861a4b6941d7366eaa6e84c8c681b4ca06ffa88a9c57c70b0fc9"} Jul 16 09:05:27.327257 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.327239 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v6mrg" event={"ID":"6c4d81b5-6013-424d-be5e-ec4877e5d16c","Type":"ContainerStarted","Data":"37dc55394a60b5c62d4b91e0925acf448644f729fc3a9764141a7f073f7c32df"} Jul 16 09:05:27.328453 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.328428 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" event={"ID":"38965c3a-8c17-4db6-af25-f28e481859dc","Type":"ContainerStarted","Data":"8b2c5f56e12b78fde4a550406ae5cdd57ea57f24d7359ef33aead2a5e0ee664e"} Jul 16 09:05:27.329448 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.329428 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wvqmn" event={"ID":"9eb0a970-f928-4d04-ab26-a2e41414db37","Type":"ContainerStarted","Data":"5c6cb2519acbcacf717f5e405f6117572b196f8310bb871ba58f966641c8438f"} Jul 16 09:05:27.330530 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.330510 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-78b27" event={"ID":"850750a5-782b-490b-b29a-ae2e05d1aff8","Type":"ContainerStarted","Data":"fada4d68935518a51c0b376f5acac2355698c385f443d1a3ca9f7969fb7b09b6"} Jul 16 09:05:27.331684 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.331662 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" event={"ID":"2615540b-77e1-4634-8099-34643fa92d7a","Type":"ContainerStarted","Data":"6a085c46474dfde358640f23ad26dc6d844364253354ffe5446365fc04c06184"} Jul 16 09:05:27.332743 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.332723 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal" event={"ID":"6a42d856b17d19cd675b76c4da5f6ebd","Type":"ContainerStarted","Data":"b89df6af1906d86203f847f48255c3b1a77f61d4d37519682a8f756d9ec4e10d"} Jul 16 09:05:27.441029 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.440996 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 09:05:27.565402 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:05:27.565370 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebfa1019_f50b_40a9_88c7_f4842be8dbff.slice/crio-1a905150d8fcfa7b4e87d60057f11f2e1bfcf2604fda4b68f42eee0d8e118dae WatchSource:0}: Error finding container 1a905150d8fcfa7b4e87d60057f11f2e1bfcf2604fda4b68f42eee0d8e118dae: Status 404 returned error can't find the container with id 1a905150d8fcfa7b4e87d60057f11f2e1bfcf2604fda4b68f42eee0d8e118dae Jul 16 09:05:27.773419 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.773386 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:27.773605 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.773473 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:27.773675 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:27.773611 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 09:05:27.773675 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:27.773671 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret podName:e84aa8e1-6582-487b-8261-61d858852378 nodeName:}" failed. No retries permitted until 2026-07-16 09:05:29.773652825 +0000 UTC m=+5.183201085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret") pod "global-pull-secret-syncer-lslxl" (UID: "e84aa8e1-6582-487b-8261-61d858852378") : object "kube-system"/"original-pull-secret" not registered Jul 16 09:05:27.774092 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:27.774072 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 09:05:27.774169 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:27.774129 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs podName:9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd nodeName:}" failed. No retries permitted until 2026-07-16 09:05:29.774113025 +0000 UTC m=+5.183661263 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs") pod "network-metrics-daemon-ph7bq" (UID: "9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 09:05:27.874028 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.873994 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfkq9\" (UniqueName: \"kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9\") pod \"network-check-target-mpfsx\" (UID: \"47a724a3-7c9b-4b6c-bd58-293cb309739f\") " pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:27.874207 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:27.874192 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 09:05:27.874271 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:27.874219 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 09:05:27.874271 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:27.874232 2567 projected.go:196] Error preparing data for projected volume kube-api-access-pfkq9 for pod openshift-network-diagnostics/network-check-target-mpfsx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 09:05:27.874376 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:27.874285 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9 podName:47a724a3-7c9b-4b6c-bd58-293cb309739f nodeName:}" failed. No retries permitted until 2026-07-16 09:05:29.874267965 +0000 UTC m=+5.283816206 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfkq9" (UniqueName: "kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9") pod "network-check-target-mpfsx" (UID: "47a724a3-7c9b-4b6c-bd58-293cb309739f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 09:05:27.896907 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:27.896880 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 09:05:28.098640 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:28.098503 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 09:00:26 +0000 UTC" deadline="2028-04-14 00:32:04.522338664 +0000 UTC" Jul 16 09:05:28.098640 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:28.098560 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15303h26m36.423782304s" Jul 16 09:05:28.319192 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:28.319161 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:28.319372 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:28.319288 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:28.319684 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:28.319667 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:28.319786 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:28.319767 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:28.344591 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:28.344503 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8glr5" event={"ID":"ebfa1019-f50b-40a9-88c7-f4842be8dbff","Type":"ContainerStarted","Data":"1a905150d8fcfa7b4e87d60057f11f2e1bfcf2604fda4b68f42eee0d8e118dae"} Jul 16 09:05:29.320871 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:29.320840 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:29.321442 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:29.320973 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:29.792491 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:29.792225 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:29.792491 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:29.792309 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:29.792491 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:29.792400 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 09:05:29.792491 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:29.792437 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 09:05:29.792491 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:29.792474 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret podName:e84aa8e1-6582-487b-8261-61d858852378 nodeName:}" failed. No retries permitted until 2026-07-16 09:05:33.792456065 +0000 UTC m=+9.202004315 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret") pod "global-pull-secret-syncer-lslxl" (UID: "e84aa8e1-6582-487b-8261-61d858852378") : object "kube-system"/"original-pull-secret" not registered Jul 16 09:05:29.792491 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:29.792493 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs podName:9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd nodeName:}" failed. No retries permitted until 2026-07-16 09:05:33.792484064 +0000 UTC m=+9.202032307 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs") pod "network-metrics-daemon-ph7bq" (UID: "9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 09:05:29.893184 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:29.893140 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfkq9\" (UniqueName: \"kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9\") pod \"network-check-target-mpfsx\" (UID: \"47a724a3-7c9b-4b6c-bd58-293cb309739f\") " pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:29.893358 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:29.893326 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 09:05:29.893358 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:29.893340 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 09:05:29.893358 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:29.893349 2567 projected.go:196] Error preparing data for projected volume kube-api-access-pfkq9 for pod openshift-network-diagnostics/network-check-target-mpfsx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 09:05:29.893521 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:29.893391 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9 podName:47a724a3-7c9b-4b6c-bd58-293cb309739f nodeName:}" failed. No retries permitted until 2026-07-16 09:05:33.893378249 +0000 UTC m=+9.302926491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfkq9" (UniqueName: "kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9") pod "network-check-target-mpfsx" (UID: "47a724a3-7c9b-4b6c-bd58-293cb309739f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 09:05:30.318281 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:30.318250 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:30.318467 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:30.318374 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:30.318761 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:30.318742 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:30.318887 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:30.318867 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:31.318877 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:31.318845 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:31.319375 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:31.318984 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:32.319211 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:32.318534 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:32.319211 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:32.318658 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:32.319211 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:32.319058 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:32.319211 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:32.319162 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:33.319409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:33.318922 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:33.319409 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:33.319058 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:33.825083 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:33.825045 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:33.825274 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:33.825125 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:33.825274 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:33.825206 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 09:05:33.825274 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:33.825231 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 09:05:33.825274 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:33.825272 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret podName:e84aa8e1-6582-487b-8261-61d858852378 nodeName:}" failed. No retries permitted until 2026-07-16 09:05:41.825252489 +0000 UTC m=+17.234800733 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret") pod "global-pull-secret-syncer-lslxl" (UID: "e84aa8e1-6582-487b-8261-61d858852378") : object "kube-system"/"original-pull-secret" not registered Jul 16 09:05:33.825500 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:33.825291 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs podName:9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd nodeName:}" failed. No retries permitted until 2026-07-16 09:05:41.82528061 +0000 UTC m=+17.234828846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs") pod "network-metrics-daemon-ph7bq" (UID: "9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 09:05:33.926078 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:33.925995 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfkq9\" (UniqueName: \"kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9\") pod \"network-check-target-mpfsx\" (UID: \"47a724a3-7c9b-4b6c-bd58-293cb309739f\") " pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:33.926255 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:33.926173 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 09:05:33.926255 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:33.926203 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 09:05:33.926255 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:33.926217 2567 projected.go:196] Error preparing data for projected volume kube-api-access-pfkq9 for pod openshift-network-diagnostics/network-check-target-mpfsx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 09:05:33.926394 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:33.926282 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9 podName:47a724a3-7c9b-4b6c-bd58-293cb309739f nodeName:}" failed. No retries permitted until 2026-07-16 09:05:41.926264076 +0000 UTC m=+17.335812332 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfkq9" (UniqueName: "kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9") pod "network-check-target-mpfsx" (UID: "47a724a3-7c9b-4b6c-bd58-293cb309739f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 09:05:34.318940 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:34.318859 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:34.319103 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:34.318859 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:34.319103 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:34.319000 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:34.319103 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:34.319078 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:35.319645 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:35.319166 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:35.319645 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:35.319303 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:36.319021 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:36.318952 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:36.319021 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:36.318988 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:36.319253 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:36.319081 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:36.319444 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:36.319410 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:37.318792 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:37.318764 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:37.319188 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:37.318886 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:38.318147 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:38.318106 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:38.318330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:38.318156 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:38.318330 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:38.318235 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:38.318429 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:38.318346 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:39.318896 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:39.318858 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:39.319358 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:39.319000 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:40.318966 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:40.318918 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:40.318966 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:40.318960 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:40.319414 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:40.319056 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:40.319414 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:40.319150 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:41.318686 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:41.318650 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:41.318906 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:41.318793 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:41.888122 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:41.888084 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:41.888555 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:41.888161 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:41.888555 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:41.888256 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 09:05:41.888555 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:41.888280 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 09:05:41.888555 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:41.888332 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs podName:9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd nodeName:}" failed. No retries permitted until 2026-07-16 09:05:57.888310918 +0000 UTC m=+33.297859174 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs") pod "network-metrics-daemon-ph7bq" (UID: "9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 09:05:41.888555 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:41.888354 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret podName:e84aa8e1-6582-487b-8261-61d858852378 nodeName:}" failed. No retries permitted until 2026-07-16 09:05:57.888343166 +0000 UTC m=+33.297891418 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret") pod "global-pull-secret-syncer-lslxl" (UID: "e84aa8e1-6582-487b-8261-61d858852378") : object "kube-system"/"original-pull-secret" not registered Jul 16 09:05:41.988595 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:41.988561 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfkq9\" (UniqueName: \"kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9\") pod \"network-check-target-mpfsx\" (UID: \"47a724a3-7c9b-4b6c-bd58-293cb309739f\") " pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:41.988773 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:41.988718 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 09:05:41.988773 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:41.988745 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 09:05:41.988773 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:41.988759 2567 projected.go:196] Error preparing data for projected volume kube-api-access-pfkq9 for pod openshift-network-diagnostics/network-check-target-mpfsx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 09:05:41.988938 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:41.988837 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9 podName:47a724a3-7c9b-4b6c-bd58-293cb309739f nodeName:}" failed. No retries permitted until 2026-07-16 09:05:57.98880496 +0000 UTC m=+33.398353213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfkq9" (UniqueName: "kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9") pod "network-check-target-mpfsx" (UID: "47a724a3-7c9b-4b6c-bd58-293cb309739f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 09:05:42.318988 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:42.318953 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:42.319176 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:42.318954 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:42.319176 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:42.319077 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:42.319176 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:42.319160 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:43.318226 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:43.318197 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:43.318650 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:43.318328 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:44.318082 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:44.318042 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:44.318272 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:44.318197 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:44.318272 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:44.318042 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:44.318686 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:44.318292 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:45.319392 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:45.319357 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:45.319781 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:45.319468 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:46.318484 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:46.318438 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:46.318484 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:46.318478 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:46.318712 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:46.318572 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:46.318764 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:46.318719 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:47.318969 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.318776 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:47.319679 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:47.319072 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:47.392066 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.392034 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-78b27" event={"ID":"850750a5-782b-490b-b29a-ae2e05d1aff8","Type":"ContainerStarted","Data":"ae3511798e6d41beb628bd167d4cc79b580b712233a77075c79cdd87b8b16c69"} Jul 16 09:05:47.393808 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.393781 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" event={"ID":"2615540b-77e1-4634-8099-34643fa92d7a","Type":"ContainerStarted","Data":"ab469e1af2000f4ef724261670706cf4cf24b46292057e9d5b2ea3cceaf05a38"} Jul 16 09:05:47.395003 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.394980 2567 generic.go:356] "Generic (PLEG): container finished" podID="6a42d856b17d19cd675b76c4da5f6ebd" containerID="b26e4275f630ab97145d961d3c568b92cfb13332fd355de854c45923408e207c" exitCode=0 Jul 16 09:05:47.395114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.395048 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal" event={"ID":"6a42d856b17d19cd675b76c4da5f6ebd","Type":"ContainerDied","Data":"b26e4275f630ab97145d961d3c568b92cfb13332fd355de854c45923408e207c"} Jul 16 09:05:47.396352 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.396333 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" event={"ID":"b51be5d2-0338-4152-9637-445cd871a7a9","Type":"ContainerStarted","Data":"3f9a879b978b7d3251977cc1b3ff0647fd2f4d9b75db6e7087699340cf192664"} Jul 16 09:05:47.397723 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.397686 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mprh4" event={"ID":"3f14599c-3b35-405b-a145-1a861e050d1c","Type":"ContainerStarted","Data":"5f7ea357135a231846572a6d5bed28f42ed828ad6e3e55883ae28d66102a69db"} Jul 16 09:05:47.399263 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.399230 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-16.ec2.internal" event={"ID":"23e3fcaf29c3b1a8898aed672fb43ec0","Type":"ContainerStarted","Data":"3b290dc3d2d391344688d11231fc7aac0c01df555b65871c19c9b919220201a0"} Jul 16 09:05:47.400942 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.400918 2567 generic.go:356] "Generic (PLEG): container finished" podID="21c0ee12-ae01-43a6-83b1-af6b69997b76" containerID="5a57f9d5a34f162f10b3f4bdf5e3436d618855ca2cfb07e9433293b1057d06cc" exitCode=0 Jul 16 09:05:47.401033 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.401000 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd79d" event={"ID":"21c0ee12-ae01-43a6-83b1-af6b69997b76","Type":"ContainerDied","Data":"5a57f9d5a34f162f10b3f4bdf5e3436d618855ca2cfb07e9433293b1057d06cc"} Jul 16 09:05:47.402651 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.402630 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v6mrg" event={"ID":"6c4d81b5-6013-424d-be5e-ec4877e5d16c","Type":"ContainerStarted","Data":"f6ce66b846cddf9a019b8cb9fa45e5572bfaeb51080b8559f096b26cf62bbaaf"} Jul 16 09:05:47.405097 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.404981 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" event={"ID":"38965c3a-8c17-4db6-af25-f28e481859dc","Type":"ContainerStarted","Data":"65dc040242e1c6372695cfdeeac1a6cf8400235fa34712ec6c2d5ae1b9b21b4e"} Jul 16 09:05:47.405097 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.405052 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" event={"ID":"38965c3a-8c17-4db6-af25-f28e481859dc","Type":"ContainerStarted","Data":"f8b727f8def8b07b32da2a539f5575d5415b410215ad2664e0749f683156f7d9"} Jul 16 09:05:47.405199 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.405104 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" event={"ID":"38965c3a-8c17-4db6-af25-f28e481859dc","Type":"ContainerStarted","Data":"b69cfa46aa90afced3224720c9eb8aa95212294ad4a0acfa9fe0897fc922a381"} Jul 16 09:05:47.406317 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.406297 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wvqmn" event={"ID":"9eb0a970-f928-4d04-ab26-a2e41414db37","Type":"ContainerStarted","Data":"1d76a8d65caab9e89a3045891cb3dfe7d95e33b559f9d74a0042efd5c642be0d"} Jul 16 09:05:47.440254 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.440211 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-78b27" podStartSLOduration=2.563438852 podStartE2EDuration="22.440198744s" totalImagesPullingTime="19.876759892s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 09:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:05:47.4152186 +0000 UTC m=+22.824766859" watchObservedRunningTime="2026-07-16 09:05:47.440198744 +0000 UTC m=+22.849747003" Jul 16 09:05:47.461925 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.461892 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mprh4" podStartSLOduration=3.022297429 podStartE2EDuration="22.461878158s" totalImagesPullingTime="19.439580729s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 09:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:05:47.441083867 +0000 UTC m=+22.850632125" watchObservedRunningTime="2026-07-16 09:05:47.461878158 +0000 UTC m=+22.871426416" Jul 16 09:05:47.461978 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.461953 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-v6mrg" podStartSLOduration=2.540421406 podStartE2EDuration="22.461949451s" totalImagesPullingTime="19.921528045s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 09:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:05:47.46122234 +0000 UTC m=+22.870770618" watchObservedRunningTime="2026-07-16 09:05:47.461949451 +0000 UTC m=+22.871497709" Jul 16 09:05:47.503505 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.503444 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-16.ec2.internal" podStartSLOduration=21.503430967 podStartE2EDuration="21.503430967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 09:05:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:05:47.503208266 +0000 UTC m=+22.912756525" watchObservedRunningTime="2026-07-16 09:05:47.503430967 +0000 UTC m=+22.912979226" Jul 16 09:05:47.520891 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.520853 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wvqmn" podStartSLOduration=2.666753865 podStartE2EDuration="22.520840413s" totalImagesPullingTime="19.854086548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:05:47.520615098 +0000 UTC m=+22.930163357" watchObservedRunningTime="2026-07-16 09:05:47.520840413 +0000 UTC m=+22.930388672" Jul 16 09:05:47.538730 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:47.538659 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gxzj8" podStartSLOduration=3.248938426 podStartE2EDuration="22.538642139s" totalImagesPullingTime="19.289703713s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 09:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:05:47.537715194 +0000 UTC m=+22.947263454" watchObservedRunningTime="2026-07-16 09:05:47.538642139 +0000 UTC m=+22.948190398" Jul 16 09:05:48.319113 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:48.318882 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:48.319570 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:48.319044 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:48.319570 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:48.319213 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:48.319570 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:48.319332 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:48.410495 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:48.410381 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8glr5" event={"ID":"ebfa1019-f50b-40a9-88c7-f4842be8dbff","Type":"ContainerStarted","Data":"b14a39aba76bfc30575ed999c1534012ed3cadb305457b5c6550e10bc9cfba07"} Jul 16 09:05:48.414694 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:48.414663 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" event={"ID":"38965c3a-8c17-4db6-af25-f28e481859dc","Type":"ContainerStarted","Data":"b15df4dad02d26976eda976019457a67a6aa0ddf93db7697e6217c07649dc679"} Jul 16 09:05:48.414859 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:48.414750 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" event={"ID":"38965c3a-8c17-4db6-af25-f28e481859dc","Type":"ContainerStarted","Data":"7a76588fb74e4c9c6203c5d66a9f5171f04bae599f3b84e4178d11dbffb7dd62"} Jul 16 09:05:48.414859 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:48.414801 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" event={"ID":"38965c3a-8c17-4db6-af25-f28e481859dc","Type":"ContainerStarted","Data":"9cb6961e4f85b547007dc0a73686b6b70c01d6ea449126c4cfc45c875639a5b1"} Jul 16 09:05:48.442997 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:48.442936 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8glr5" podStartSLOduration=4.191057615 podStartE2EDuration="23.442914181s" totalImagesPullingTime="19.251856566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 09:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:05:48.426662337 +0000 UTC m=+23.836210615" watchObservedRunningTime="2026-07-16 09:05:48.442914181 +0000 UTC m=+23.852462442" Jul 16 09:05:48.932766 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:48.932720 2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 09:05:49.152720 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:49.152465 2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T09:05:48.93274124Z","UUID":"4ca887e2-df3d-4afc-be01-33800a19125c","Handler":null,"Name":"","Endpoint":""} Jul 16 09:05:49.154116 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:49.154083 2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 09:05:49.154245 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:49.154128 2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 09:05:49.169109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:49.169080 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wvqmn" Jul 16 09:05:49.169639 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:49.169622 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wvqmn" Jul 16 09:05:49.318109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:49.318079 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:49.318362 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:49.318214 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:49.419929 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:49.419846 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" event={"ID":"2615540b-77e1-4634-8099-34643fa92d7a","Type":"ContainerStarted","Data":"88eb26a143f084a62514f779283c8118b00fe92a20df19c125f879b251a98055"} Jul 16 09:05:49.422389 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:49.422364 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal" event={"ID":"6a42d856b17d19cd675b76c4da5f6ebd","Type":"ContainerStarted","Data":"d475c79f4c267800d16aee2397b71c499b8434d3d49436c81590ca0cf5a4f5ce"} Jul 16 09:05:49.422865 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:49.422844 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wvqmn" Jul 16 09:05:49.423117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:49.423099 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wvqmn" Jul 16 09:05:49.440412 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:49.440368 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-16.ec2.internal" podStartSLOduration=23.44035379 podStartE2EDuration="23.44035379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 09:05:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:05:49.439983524 +0000 UTC m=+24.849531801" watchObservedRunningTime="2026-07-16 09:05:49.44035379 +0000 UTC m=+24.849902050" Jul 16 09:05:50.318764 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:50.318725 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:50.318764 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:50.318750 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:50.319014 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:50.318857 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:50.319072 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:50.319010 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:50.427671 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:50.427629 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" event={"ID":"38965c3a-8c17-4db6-af25-f28e481859dc","Type":"ContainerStarted","Data":"ec9a3cd19493aab2d08cfea889df11fd9a7727ab9da61b58c6f09b707b212da7"} Jul 16 09:05:51.318745 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:51.318710 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:51.318945 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:51.318865 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:52.318470 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:52.318307 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:52.319032 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:52.318307 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:52.319032 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:52.318591 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:52.319032 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:52.318610 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:52.434475 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:52.434406 2567 generic.go:356] "Generic (PLEG): container finished" podID="21c0ee12-ae01-43a6-83b1-af6b69997b76" containerID="4f0b0e2226d273b6d2db0f5f4a313f4950efb395471fc8657dec442df343d62d" exitCode=0 Jul 16 09:05:52.434643 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:52.434496 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd79d" event={"ID":"21c0ee12-ae01-43a6-83b1-af6b69997b76","Type":"ContainerDied","Data":"4f0b0e2226d273b6d2db0f5f4a313f4950efb395471fc8657dec442df343d62d"} Jul 16 09:05:52.438473 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:52.438375 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" event={"ID":"38965c3a-8c17-4db6-af25-f28e481859dc","Type":"ContainerStarted","Data":"99bd23ae496c314f2ea0db7462df8a377733b595d3e35279e034cd660105f67d"} Jul 16 09:05:52.438607 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:52.438595 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:52.440614 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:52.440594 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" event={"ID":"2615540b-77e1-4634-8099-34643fa92d7a","Type":"ContainerStarted","Data":"ecbd0f0941b6ecd0eac2146819cae064fdb14ebdea2768c00ce792b852439224"} Jul 16 09:05:52.453770 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:52.453752 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:52.484572 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:52.484531 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhcj" podStartSLOduration=3.19397578 podStartE2EDuration="27.484518499s" totalImagesPullingTime="24.290542719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 09:05:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:05:52.483989319 +0000 UTC m=+27.893537577" watchObservedRunningTime="2026-07-16 09:05:52.484518499 +0000 UTC m=+27.894066793" Jul 16 09:05:52.517916 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:52.517859 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" podStartSLOduration=7.647042274 podStartE2EDuration="27.517810644s" totalImagesPullingTime="19.87076837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 09:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:05:52.514680704 +0000 UTC m=+27.924228963" watchObservedRunningTime="2026-07-16 09:05:52.517810644 +0000 UTC m=+27.927358904" Jul 16 09:05:53.318797 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:53.318756 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:53.319540 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:53.318904 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:53.443243 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:53.443189 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:53.443530 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:53.443515 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:53.456941 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:53.456914 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:05:54.183265 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:54.183043 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mpfsx"] Jul 16 09:05:54.183480 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:54.183341 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:54.183480 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:54.183419 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:54.186038 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:54.186011 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ph7bq"] Jul 16 09:05:54.186184 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:54.186137 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:54.186274 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:54.186253 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:54.186562 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:54.186543 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lslxl"] Jul 16 09:05:54.186623 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:54.186614 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:54.186701 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:54.186684 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:54.447274 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:54.447244 2567 generic.go:356] "Generic (PLEG): container finished" podID="21c0ee12-ae01-43a6-83b1-af6b69997b76" containerID="e447762f263986c074dd0cf49699d65abf68a7ec7d344f6ac784591010ad62b2" exitCode=0 Jul 16 09:05:54.447880 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:54.447311 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd79d" event={"ID":"21c0ee12-ae01-43a6-83b1-af6b69997b76","Type":"ContainerDied","Data":"e447762f263986c074dd0cf49699d65abf68a7ec7d344f6ac784591010ad62b2"} Jul 16 09:05:56.318976 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:56.318947 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:56.319303 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:56.319043 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:56.319303 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:56.319048 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:56.319303 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:56.319050 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:56.319303 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:56.319131 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:56.319303 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:56.319193 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:56.453423 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:56.453338 2567 generic.go:356] "Generic (PLEG): container finished" podID="21c0ee12-ae01-43a6-83b1-af6b69997b76" containerID="e485ead7f4250f8d4b3bb7ad34af36a4b3d09a485dd25bc4baf5c045ede229bb" exitCode=0 Jul 16 09:05:56.453423 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:56.453378 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd79d" event={"ID":"21c0ee12-ae01-43a6-83b1-af6b69997b76","Type":"ContainerDied","Data":"e485ead7f4250f8d4b3bb7ad34af36a4b3d09a485dd25bc4baf5c045ede229bb"} Jul 16 09:05:57.908474 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:57.908439 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:57.908977 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:57.908503 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:57.908977 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:57.908600 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 09:05:57.908977 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:57.908667 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret podName:e84aa8e1-6582-487b-8261-61d858852378 nodeName:}" failed. No retries permitted until 2026-07-16 09:06:29.908652148 +0000 UTC m=+65.318200386 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret") pod "global-pull-secret-syncer-lslxl" (UID: "e84aa8e1-6582-487b-8261-61d858852378") : object "kube-system"/"original-pull-secret" not registered Jul 16 09:05:57.908977 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:57.908604 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 09:05:57.908977 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:57.908758 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs podName:9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd nodeName:}" failed. No retries permitted until 2026-07-16 09:06:29.908739213 +0000 UTC m=+65.318287461 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs") pod "network-metrics-daemon-ph7bq" (UID: "9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 09:05:58.009751 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:58.009717 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfkq9\" (UniqueName: \"kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9\") pod \"network-check-target-mpfsx\" (UID: \"47a724a3-7c9b-4b6c-bd58-293cb309739f\") " pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:58.009902 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:58.009883 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 09:05:58.009902 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:58.009899 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 09:05:58.010031 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:58.009908 2567 projected.go:196] Error preparing data for projected volume kube-api-access-pfkq9 for pod openshift-network-diagnostics/network-check-target-mpfsx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 09:05:58.010031 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:58.009961 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9 podName:47a724a3-7c9b-4b6c-bd58-293cb309739f nodeName:}" failed. No retries permitted until 2026-07-16 09:06:30.009944509 +0000 UTC m=+65.419492747 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfkq9" (UniqueName: "kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9") pod "network-check-target-mpfsx" (UID: "47a724a3-7c9b-4b6c-bd58-293cb309739f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 09:05:58.319072 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:58.319034 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:05:58.319239 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:58.319079 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:05:58.319239 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:58.319079 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:05:58.319239 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:58.319158 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mpfsx" podUID="47a724a3-7c9b-4b6c-bd58-293cb309739f" Jul 16 09:05:58.319383 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:58.319271 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:05:58.319424 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:05:58.319390 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lslxl" podUID="e84aa8e1-6582-487b-8261-61d858852378" Jul 16 09:05:59.849185 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.849157 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-16.ec2.internal" event="NodeReady" Jul 16 09:05:59.849581 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.849295 2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 09:05:59.901865 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.901835 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-szwcd"] Jul 16 09:05:59.936388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.936360 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kzpnm"] Jul 16 09:05:59.936565 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.936500 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-szwcd" Jul 16 09:05:59.939214 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.939193 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 09:05:59.939340 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.939223 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 09:05:59.939340 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.939289 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xmnzs\"" Jul 16 09:05:59.954189 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.954171 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-szwcd"] Jul 16 09:05:59.954278 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.954209 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kzpnm"] Jul 16 09:05:59.954331 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.954275 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kzpnm" Jul 16 09:05:59.956883 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.956866 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q4lck\"" Jul 16 09:05:59.956977 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.956886 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 09:05:59.956977 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.956906 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 09:05:59.956977 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:05:59.956914 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 09:06:00.022755 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.022721 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvhsm\" (UniqueName: \"kubernetes.io/projected/d2d206a0-1603-4fd8-936a-dacba9979e0c-kube-api-access-mvhsm\") pod \"ingress-canary-kzpnm\" (UID: \"d2d206a0-1603-4fd8-936a-dacba9979e0c\") " pod="openshift-ingress-canary/ingress-canary-kzpnm" Jul 16 09:06:00.022908 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.022761 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:00.022908 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.022844 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d2d206a0-1603-4fd8-936a-dacba9979e0c-cert\") pod \"ingress-canary-kzpnm\" (UID: \"d2d206a0-1603-4fd8-936a-dacba9979e0c\") " pod="openshift-ingress-canary/ingress-canary-kzpnm" Jul 16 09:06:00.022908 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.022869 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1ef4d9f-e004-434d-af44-a51c96bfeb32-tmp-dir\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:00.023041 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.022968 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ef4d9f-e004-434d-af44-a51c96bfeb32-config-volume\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:00.023041 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.023000 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqx9f\" (UniqueName: \"kubernetes.io/projected/f1ef4d9f-e004-434d-af44-a51c96bfeb32-kube-api-access-wqx9f\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:00.123901 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.123803 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvhsm\" (UniqueName: \"kubernetes.io/projected/d2d206a0-1603-4fd8-936a-dacba9979e0c-kube-api-access-mvhsm\") pod \"ingress-canary-kzpnm\" (UID: \"d2d206a0-1603-4fd8-936a-dacba9979e0c\") " pod="openshift-ingress-canary/ingress-canary-kzpnm" Jul 16 09:06:00.123901 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.123862 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:00.123901 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.123902 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d2d206a0-1603-4fd8-936a-dacba9979e0c-cert\") pod \"ingress-canary-kzpnm\" (UID: \"d2d206a0-1603-4fd8-936a-dacba9979e0c\") " pod="openshift-ingress-canary/ingress-canary-kzpnm" Jul 16 09:06:00.124153 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.123918 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1ef4d9f-e004-434d-af44-a51c96bfeb32-tmp-dir\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:00.124208 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:00.124149 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 09:06:00.124208 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:00.124207 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls podName:f1ef4d9f-e004-434d-af44-a51c96bfeb32 nodeName:}" failed. No retries permitted until 2026-07-16 09:06:00.624188746 +0000 UTC m=+36.033736996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls") pod "dns-default-szwcd" (UID: "f1ef4d9f-e004-434d-af44-a51c96bfeb32") : secret "dns-default-metrics-tls" not found Jul 16 09:06:00.124317 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.124278 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ef4d9f-e004-434d-af44-a51c96bfeb32-config-volume\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:00.124372 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.124314 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqx9f\" (UniqueName: \"kubernetes.io/projected/f1ef4d9f-e004-434d-af44-a51c96bfeb32-kube-api-access-wqx9f\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:00.124577 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.124548 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f1ef4d9f-e004-434d-af44-a51c96bfeb32-tmp-dir\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:00.124804 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.124786 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f1ef4d9f-e004-434d-af44-a51c96bfeb32-config-volume\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:00.128693 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.128578 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d2d206a0-1603-4fd8-936a-dacba9979e0c-cert\") pod \"ingress-canary-kzpnm\" (UID: \"d2d206a0-1603-4fd8-936a-dacba9979e0c\") " pod="openshift-ingress-canary/ingress-canary-kzpnm" Jul 16 09:06:00.133273 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.133253 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvhsm\" (UniqueName: \"kubernetes.io/projected/d2d206a0-1603-4fd8-936a-dacba9979e0c-kube-api-access-mvhsm\") pod \"ingress-canary-kzpnm\" (UID: \"d2d206a0-1603-4fd8-936a-dacba9979e0c\") " pod="openshift-ingress-canary/ingress-canary-kzpnm" Jul 16 09:06:00.133400 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.133374 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqx9f\" (UniqueName: \"kubernetes.io/projected/f1ef4d9f-e004-434d-af44-a51c96bfeb32-kube-api-access-wqx9f\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:00.263442 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.263412 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kzpnm" Jul 16 09:06:00.318975 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.318950 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:06:00.319109 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.318984 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:06:00.319265 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.319238 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:06:00.322670 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.322234 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xwv7m\"" Jul 16 09:06:00.322670 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.322295 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 09:06:00.322898 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.322831 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 09:06:00.323097 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.323077 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h777f\"" Jul 16 09:06:00.323171 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.323094 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 09:06:00.323342 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.323324 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 09:06:00.627338 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.627313 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:00.627468 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:00.627429 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 09:06:00.627507 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:00.627485 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls podName:f1ef4d9f-e004-434d-af44-a51c96bfeb32 nodeName:}" failed. No retries permitted until 2026-07-16 09:06:01.627468323 +0000 UTC m=+37.037016559 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls") pod "dns-default-szwcd" (UID: "f1ef4d9f-e004-434d-af44-a51c96bfeb32") : secret "dns-default-metrics-tls" not found Jul 16 09:06:00.641706 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:00.641677 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kzpnm"] Jul 16 09:06:00.702357 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:06:00.702325 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2d206a0_1603_4fd8_936a_dacba9979e0c.slice/crio-d00b2fc8a6210bb1d8095b4bcf7ee89c9be46d8126d4e26dc5a842b59de350b0 WatchSource:0}: Error finding container d00b2fc8a6210bb1d8095b4bcf7ee89c9be46d8126d4e26dc5a842b59de350b0: Status 404 returned error can't find the container with id d00b2fc8a6210bb1d8095b4bcf7ee89c9be46d8126d4e26dc5a842b59de350b0 Jul 16 09:06:01.464786 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:01.464751 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kzpnm" event={"ID":"d2d206a0-1603-4fd8-936a-dacba9979e0c","Type":"ContainerStarted","Data":"d00b2fc8a6210bb1d8095b4bcf7ee89c9be46d8126d4e26dc5a842b59de350b0"} Jul 16 09:06:01.467210 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:01.467186 2567 generic.go:356] "Generic (PLEG): container finished" podID="21c0ee12-ae01-43a6-83b1-af6b69997b76" containerID="becd89cd546909571b45371d2200b7e7940479a2ca7d90a264c49abed272718f" exitCode=0 Jul 16 09:06:01.467330 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:01.467220 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd79d" event={"ID":"21c0ee12-ae01-43a6-83b1-af6b69997b76","Type":"ContainerDied","Data":"becd89cd546909571b45371d2200b7e7940479a2ca7d90a264c49abed272718f"} Jul 16 09:06:01.633390 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:01.633359 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:01.633549 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:01.633528 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 09:06:01.633619 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:01.633607 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls podName:f1ef4d9f-e004-434d-af44-a51c96bfeb32 nodeName:}" failed. No retries permitted until 2026-07-16 09:06:03.633587025 +0000 UTC m=+39.043135278 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls") pod "dns-default-szwcd" (UID: "f1ef4d9f-e004-434d-af44-a51c96bfeb32") : secret "dns-default-metrics-tls" not found Jul 16 09:06:02.473382 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:02.473348 2567 generic.go:356] "Generic (PLEG): container finished" podID="21c0ee12-ae01-43a6-83b1-af6b69997b76" containerID="38515f92d6ceb3dff42a5f2f6a775246f4e3b249a6730443d06c2b64ea93cfe3" exitCode=0 Jul 16 09:06:02.473757 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:02.473434 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd79d" event={"ID":"21c0ee12-ae01-43a6-83b1-af6b69997b76","Type":"ContainerDied","Data":"38515f92d6ceb3dff42a5f2f6a775246f4e3b249a6730443d06c2b64ea93cfe3"} Jul 16 09:06:03.481130 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:03.480909 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sd79d" event={"ID":"21c0ee12-ae01-43a6-83b1-af6b69997b76","Type":"ContainerStarted","Data":"c389e1742fdc0fe939be9c3cfd2a8a56e466dd59e46b0a636f5860282cf344db"} Jul 16 09:06:03.508740 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:03.508677 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-sd79d" podStartSLOduration=7.82605854 podStartE2EDuration="38.508655228s" totalImagesPullingTime="30.682596688s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 09:05:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:06:03.506887344 +0000 UTC m=+38.916435638" watchObservedRunningTime="2026-07-16 09:06:03.508655228 +0000 UTC m=+38.918203487" Jul 16 09:06:03.650608 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:03.650584 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:03.650721 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:03.650701 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 09:06:03.650783 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:03.650772 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls podName:f1ef4d9f-e004-434d-af44-a51c96bfeb32 nodeName:}" failed. No retries permitted until 2026-07-16 09:06:07.650751582 +0000 UTC m=+43.060299833 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls") pod "dns-default-szwcd" (UID: "f1ef4d9f-e004-434d-af44-a51c96bfeb32") : secret "dns-default-metrics-tls" not found Jul 16 09:06:04.485151 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:04.485114 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kzpnm" event={"ID":"d2d206a0-1603-4fd8-936a-dacba9979e0c","Type":"ContainerStarted","Data":"e3c0d48e9dd77411509d773720704e230e7a9351a2b308831ddefa1c9b51609a"} Jul 16 09:06:04.503452 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:04.503407 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kzpnm" podStartSLOduration=2.591221965 podStartE2EDuration="5.503391758s" totalImagesPullingTime="2.912169793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:05:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:06:04.502129522 +0000 UTC m=+39.911677779" watchObservedRunningTime="2026-07-16 09:06:04.503391758 +0000 UTC m=+39.912940046" Jul 16 09:06:07.679637 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:07.679603 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:07.680090 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:07.679750 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 09:06:07.680090 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:07.679810 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls podName:f1ef4d9f-e004-434d-af44-a51c96bfeb32 nodeName:}" failed. No retries permitted until 2026-07-16 09:06:15.679795095 +0000 UTC m=+51.089343332 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls") pod "dns-default-szwcd" (UID: "f1ef4d9f-e004-434d-af44-a51c96bfeb32") : secret "dns-default-metrics-tls" not found Jul 16 09:06:15.732402 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:15.732361 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:15.732897 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:15.732486 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 09:06:15.732897 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:15.732550 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls podName:f1ef4d9f-e004-434d-af44-a51c96bfeb32 nodeName:}" failed. No retries permitted until 2026-07-16 09:06:31.732532272 +0000 UTC m=+67.142080510 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls") pod "dns-default-szwcd" (UID: "f1ef4d9f-e004-434d-af44-a51c96bfeb32") : secret "dns-default-metrics-tls" not found Jul 16 09:06:25.459389 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:25.459361 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qqw5z" Jul 16 09:06:29.929284 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:29.929246 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:06:29.929689 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:29.929297 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:06:29.933414 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:29.933396 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 09:06:29.933497 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:29.933482 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 09:06:29.940360 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:29.940342 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 09:06:29.940426 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:29.940416 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs podName:9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd nodeName:}" failed. No retries permitted until 2026-07-16 09:07:33.940402007 +0000 UTC m=+129.349950245 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs") pod "network-metrics-daemon-ph7bq" (UID: "9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd") : secret "metrics-daemon-secret" not found Jul 16 09:06:29.943393 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:29.943371 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e84aa8e1-6582-487b-8261-61d858852378-original-pull-secret\") pod \"global-pull-secret-syncer-lslxl\" (UID: \"e84aa8e1-6582-487b-8261-61d858852378\") " pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:06:30.030552 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:30.030524 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfkq9\" (UniqueName: \"kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9\") pod \"network-check-target-mpfsx\" (UID: \"47a724a3-7c9b-4b6c-bd58-293cb309739f\") " pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:06:30.033852 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:30.033835 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 09:06:30.038444 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:30.038431 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lslxl" Jul 16 09:06:30.043619 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:30.043604 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 09:06:30.054444 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:30.054423 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfkq9\" (UniqueName: \"kubernetes.io/projected/47a724a3-7c9b-4b6c-bd58-293cb309739f-kube-api-access-pfkq9\") pod \"network-check-target-mpfsx\" (UID: \"47a724a3-7c9b-4b6c-bd58-293cb309739f\") " pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:06:30.156393 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:30.156368 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lslxl"] Jul 16 09:06:30.159848 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:06:30.159807 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode84aa8e1_6582_487b_8261_61d858852378.slice/crio-d8dc6ebbf11fb9de1db3ccd9d417bc717fd4a37722a325894a9801f322744304 WatchSource:0}: Error finding container d8dc6ebbf11fb9de1db3ccd9d417bc717fd4a37722a325894a9801f322744304: Status 404 returned error can't find the container with id d8dc6ebbf11fb9de1db3ccd9d417bc717fd4a37722a325894a9801f322744304 Jul 16 09:06:30.334940 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:30.334916 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xwv7m\"" Jul 16 09:06:30.342215 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:30.342194 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:06:30.455427 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:30.455393 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mpfsx"] Jul 16 09:06:30.459122 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:06:30.459086 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47a724a3_7c9b_4b6c_bd58_293cb309739f.slice/crio-611c59612bcab0846f7c74d11dcaf370b9e932a8eb97495c155f0dbd55958448 WatchSource:0}: Error finding container 611c59612bcab0846f7c74d11dcaf370b9e932a8eb97495c155f0dbd55958448: Status 404 returned error can't find the container with id 611c59612bcab0846f7c74d11dcaf370b9e932a8eb97495c155f0dbd55958448 Jul 16 09:06:30.542651 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:30.542610 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mpfsx" event={"ID":"47a724a3-7c9b-4b6c-bd58-293cb309739f","Type":"ContainerStarted","Data":"611c59612bcab0846f7c74d11dcaf370b9e932a8eb97495c155f0dbd55958448"} Jul 16 09:06:30.543610 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:30.543587 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lslxl" event={"ID":"e84aa8e1-6582-487b-8261-61d858852378","Type":"ContainerStarted","Data":"d8dc6ebbf11fb9de1db3ccd9d417bc717fd4a37722a325894a9801f322744304"} Jul 16 09:06:31.744712 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:31.744068 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:06:31.744712 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:31.744238 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 09:06:31.744712 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:06:31.744317 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls podName:f1ef4d9f-e004-434d-af44-a51c96bfeb32 nodeName:}" failed. No retries permitted until 2026-07-16 09:07:03.744296523 +0000 UTC m=+99.153844774 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls") pod "dns-default-szwcd" (UID: "f1ef4d9f-e004-434d-af44-a51c96bfeb32") : secret "dns-default-metrics-tls" not found Jul 16 09:06:35.558633 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:35.558597 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lslxl" event={"ID":"e84aa8e1-6582-487b-8261-61d858852378","Type":"ContainerStarted","Data":"02a63b52d6d1e7e77ec82a2c14a0fec9ff95401803aa64c3b89c3e066f408dd3"} Jul 16 09:06:35.559999 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:35.559963 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mpfsx" event={"ID":"47a724a3-7c9b-4b6c-bd58-293cb309739f","Type":"ContainerStarted","Data":"6af899c3d2c0277bb581acb500a94b17f2271ff4b2ea977d8b22303c000333a3"} Jul 16 09:06:35.560119 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:35.560076 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:06:35.591195 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:35.591147 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lslxl" podStartSLOduration=65.871870256 podStartE2EDuration="1m10.591132412s" totalImagesPullingTime="4.719262156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 09:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:06:35.575589073 +0000 UTC m=+70.985137332" watchObservedRunningTime="2026-07-16 09:06:35.591132412 +0000 UTC m=+71.000680670" Jul 16 09:06:35.591556 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:06:35.591536 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mpfsx" podStartSLOduration=66.181204846 podStartE2EDuration="1m10.591531244s" totalImagesPullingTime="4.410326398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:06:35.590447242 +0000 UTC m=+70.999995500" watchObservedRunningTime="2026-07-16 09:06:35.591531244 +0000 UTC m=+71.001079503" Jul 16 09:07:03.758831 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:07:03.758786 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:07:03.759258 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:07:03.758937 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 09:07:03.759258 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:07:03.759008 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls podName:f1ef4d9f-e004-434d-af44-a51c96bfeb32 nodeName:}" failed. No retries permitted until 2026-07-16 09:08:07.758990371 +0000 UTC m=+163.168538611 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls") pod "dns-default-szwcd" (UID: "f1ef4d9f-e004-434d-af44-a51c96bfeb32") : secret "dns-default-metrics-tls" not found Jul 16 09:07:06.564582 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:07:06.564550 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mpfsx" Jul 16 09:07:33.960581 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:07:33.960536 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:07:33.961100 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:07:33.960712 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 09:07:33.961100 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:07:33.960781 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs podName:9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd nodeName:}" failed. No retries permitted until 2026-07-16 09:09:35.96076463 +0000 UTC m=+251.370312872 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs") pod "network-metrics-daemon-ph7bq" (UID: "9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd") : secret "metrics-daemon-secret" not found Jul 16 09:08:02.946321 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:08:02.946266 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-szwcd" podUID="f1ef4d9f-e004-434d-af44-a51c96bfeb32" Jul 16 09:08:03.343546 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:08:03.343504 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:08:03.759525 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:08:03.759498 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-szwcd" Jul 16 09:08:07.789644 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:08:07.789613 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:08:07.790067 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:08:07.789729 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 09:08:07.790067 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:08:07.789792 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls podName:f1ef4d9f-e004-434d-af44-a51c96bfeb32 nodeName:}" failed. No retries permitted until 2026-07-16 09:10:09.789775875 +0000 UTC m=+285.199324113 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls") pod "dns-default-szwcd" (UID: "f1ef4d9f-e004-434d-af44-a51c96bfeb32") : secret "dns-default-metrics-tls" not found Jul 16 09:08:16.318721 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:08:16.318688 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:09:20.574682 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.574648 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-f4qh9"] Jul 16 09:09:20.576572 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.576556 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-z95sx"] Jul 16 09:09:20.576700 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.576683 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:20.578027 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.578011 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9"] Jul 16 09:09:20.578143 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.578126 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-z95sx" Jul 16 09:09:20.579433 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.579420 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25"] Jul 16 09:09:20.579528 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.579516 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" Jul 16 09:09:20.580988 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.580461 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 16 09:09:20.581449 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.581395 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/42c8464a-45ad-406d-b06a-cc2e8c6d1cbc-trusted-ca\") pod \"console-operator-7866497494-f4qh9\" (UID: \"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc\") " pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:20.581703 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.581685 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42c8464a-45ad-406d-b06a-cc2e8c6d1cbc-serving-cert\") pod \"console-operator-7866497494-f4qh9\" (UID: \"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc\") " pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:20.582550 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.581830 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 16 09:09:20.582550 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.581995 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42c8464a-45ad-406d-b06a-cc2e8c6d1cbc-config\") pod \"console-operator-7866497494-f4qh9\" (UID: \"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc\") " pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:20.582771 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.582755 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 16 09:09:20.582892 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.582875 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 16 09:09:20.582963 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:20.582884 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-134-16.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-cluster-samples-operator\": no relationship found between node 'ip-10-0-134-16.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 16 09:09:20.583074 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.583051 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-gwmzl\"" Jul 16 09:09:20.583230 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:20.583202 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"samples-operator-tls\" is forbidden: User \"system:node:ip-10-0-134-16.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-cluster-samples-operator\": no relationship found between node 'ip-10-0-134-16.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" type="*v1.Secret" Jul 16 09:09:20.583470 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:20.583444 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-samples-operator-dockercfg-dqt7g\" is forbidden: User \"system:node:ip-10-0-134-16.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-cluster-samples-operator\": no relationship found between node 'ip-10-0-134-16.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-dqt7g\"" type="*v1.Secret" Jul 16 09:09:20.583550 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.583471 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-xqqv6"] Jul 16 09:09:20.583666 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.583645 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:20.583794 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:20.583768 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-134-16.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-cluster-samples-operator\": no relationship found between node 'ip-10-0-134-16.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 16 09:09:20.583879 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.583843 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-9k2n6\"" Jul 16 09:09:20.586741 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.586721 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6cz65\"" Jul 16 09:09:20.586966 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.586948 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" Jul 16 09:09:20.587431 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.587415 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 09:09:20.588074 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.588059 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 16 09:09:20.588311 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.588295 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 16 09:09:20.588382 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.588326 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 09:09:20.589223 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.589205 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 16 09:09:20.590911 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.590897 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 16 09:09:20.592708 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.592692 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-m6lv5\"" Jul 16 09:09:20.592890 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.592694 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 16 09:09:20.594686 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.594669 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-z95sx"] Jul 16 09:09:20.605043 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.605024 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-xqqv6"] Jul 16 09:09:20.607654 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.607635 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9"] Jul 16 09:09:20.615737 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.615717 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-f4qh9"] Jul 16 09:09:20.616421 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.616403 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25"] Jul 16 09:09:20.678853 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.678805 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-5gbht"] Jul 16 09:09:20.680927 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.680912 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.683458 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.683435 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf55m\" (UniqueName: \"kubernetes.io/projected/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-kube-api-access-bf55m\") pod \"cluster-samples-operator-65db858894-fvgz9\" (UID: \"93e8fe28-842d-45a7-a34a-ccc2fa3ecf31\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" Jul 16 09:09:20.683577 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.683495 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/42c8464a-45ad-406d-b06a-cc2e8c6d1cbc-trusted-ca\") pod \"console-operator-7866497494-f4qh9\" (UID: \"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc\") " pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:20.683577 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.683548 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42c8464a-45ad-406d-b06a-cc2e8c6d1cbc-serving-cert\") pod \"console-operator-7866497494-f4qh9\" (UID: \"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc\") " pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:20.683691 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.683605 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8pqh\" (UniqueName: \"kubernetes.io/projected/42c8464a-45ad-406d-b06a-cc2e8c6d1cbc-kube-api-access-v8pqh\") pod \"console-operator-7866497494-f4qh9\" (UID: \"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc\") " pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:20.683691 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.683644 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42c8464a-45ad-406d-b06a-cc2e8c6d1cbc-config\") pod \"console-operator-7866497494-f4qh9\" (UID: \"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc\") " pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:20.683870 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.683808 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmhns\" (UniqueName: \"kubernetes.io/projected/4449268e-1a20-4756-9d02-8939148e56ca-kube-api-access-vmhns\") pod \"network-check-source-869dcfcdcd-z95sx\" (UID: \"4449268e-1a20-4756-9d02-8939148e56ca\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-z95sx" Jul 16 09:09:20.683870 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.683863 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fvgz9\" (UID: \"93e8fe28-842d-45a7-a34a-ccc2fa3ecf31\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" Jul 16 09:09:20.684129 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.684105 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 16 09:09:20.684234 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.684116 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zhjs7\"" Jul 16 09:09:20.684234 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.684168 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 09:09:20.684405 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.684382 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/42c8464a-45ad-406d-b06a-cc2e8c6d1cbc-trusted-ca\") pod \"console-operator-7866497494-f4qh9\" (UID: \"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc\") " pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:20.684466 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.684453 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42c8464a-45ad-406d-b06a-cc2e8c6d1cbc-config\") pod \"console-operator-7866497494-f4qh9\" (UID: \"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc\") " pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:20.684522 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.684470 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 16 09:09:20.684522 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.684508 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 09:09:20.686018 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.686000 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42c8464a-45ad-406d-b06a-cc2e8c6d1cbc-serving-cert\") pod \"console-operator-7866497494-f4qh9\" (UID: \"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc\") " pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:20.692773 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.692118 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 16 09:09:20.693437 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.693418 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m"] Jul 16 09:09:20.695209 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.695193 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" Jul 16 09:09:20.695447 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.695428 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b5b8d48f6-r62kk"] Jul 16 09:09:20.696996 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.696982 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-zfjdj"] Jul 16 09:09:20.697094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.697080 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.698615 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.698599 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-zfjdj" Jul 16 09:09:20.698964 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.698951 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 16 09:09:20.699223 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.699205 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 16 09:09:20.699307 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.699226 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 16 09:09:20.699784 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.699770 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-kkc82\"" Jul 16 09:09:20.700003 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.699989 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 16 09:09:20.703793 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.703771 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8dsdg\"" Jul 16 09:09:20.705840 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.704095 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 09:09:20.705840 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.704415 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 16 09:09:20.705840 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.704800 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-5gbht"] Jul 16 09:09:20.706779 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.706761 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 09:09:20.706949 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.706929 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-vj6sn\"" Jul 16 09:09:20.707310 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.707290 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 09:09:20.708036 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.708017 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 16 09:09:20.711031 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.711010 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 09:09:20.715363 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.715344 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m"] Jul 16 09:09:20.718351 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.718305 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-zfjdj"] Jul 16 09:09:20.737086 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.737060 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b5b8d48f6-r62kk"] Jul 16 09:09:20.785216 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785191 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8pqh\" (UniqueName: \"kubernetes.io/projected/42c8464a-45ad-406d-b06a-cc2e8c6d1cbc-kube-api-access-v8pqh\") pod \"console-operator-7866497494-f4qh9\" (UID: \"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc\") " pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:20.785355 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785221 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-bound-sa-token\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.785355 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785241 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4cf179b-d79d-44c6-a486-6cc872cc84ff-ca-trust-extracted\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.785355 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785342 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4cf179b-d79d-44c6-a486-6cc872cc84ff-installation-pull-secrets\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.785453 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785373 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcfe3047-1320-4d55-adb2-636de23660c4-serving-cert\") pod \"service-ca-operator-7695fc6c8c-kz47m\" (UID: \"dcfe3047-1320-4d55-adb2-636de23660c4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" Jul 16 09:09:20.785453 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785391 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa2a7c4-5499-408f-9c98-acc1debc7be0-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.785453 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785412 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkp68\" (UniqueName: \"kubernetes.io/projected/dcfe3047-1320-4d55-adb2-636de23660c4-kube-api-access-bkp68\") pod \"service-ca-operator-7695fc6c8c-kz47m\" (UID: \"dcfe3047-1320-4d55-adb2-636de23660c4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" Jul 16 09:09:20.785539 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785453 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9fa2a7c4-5499-408f-9c98-acc1debc7be0-snapshots\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.785539 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785474 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4cf179b-d79d-44c6-a486-6cc872cc84ff-image-registry-private-configuration\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.785539 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785527 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-xqqv6\" (UID: \"7d93ba98-9730-4d8e-9ab5-d5111a98387c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" Jul 16 09:09:20.785638 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785546 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa2a7c4-5499-408f-9c98-acc1debc7be0-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.785638 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785560 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptck5\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-kube-api-access-ptck5\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.785638 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785579 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ws9t\" (UniqueName: \"kubernetes.io/projected/8322ecd6-180e-462a-97d2-8cd3942f208d-kube-api-access-2ws9t\") pod \"volume-data-source-validator-78495597f6-zfjdj\" (UID: \"8322ecd6-180e-462a-97d2-8cd3942f208d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-zfjdj" Jul 16 09:09:20.785638 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785618 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.785751 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785640 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dcfe3047-1320-4d55-adb2-636de23660c4-config\") pod \"service-ca-operator-7695fc6c8c-kz47m\" (UID: \"dcfe3047-1320-4d55-adb2-636de23660c4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" Jul 16 09:09:20.785751 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785680 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-certificates\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.785751 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785696 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d93ba98-9730-4d8e-9ab5-d5111a98387c-nginx-conf\") pod \"networking-console-plugin-6cf856758c-xqqv6\" (UID: \"7d93ba98-9730-4d8e-9ab5-d5111a98387c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" Jul 16 09:09:20.785751 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785721 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdsxt\" (UniqueName: \"kubernetes.io/projected/9fa2a7c4-5499-408f-9c98-acc1debc7be0-kube-api-access-bdsxt\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.785751 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785738 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/320437bb-7863-4d89-8736-4c69fe673ad5-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:20.785915 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785775 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl7dw\" (UniqueName: \"kubernetes.io/projected/320437bb-7863-4d89-8736-4c69fe673ad5-kube-api-access-hl7dw\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:20.785915 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785802 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4cf179b-d79d-44c6-a486-6cc872cc84ff-trusted-ca\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.785915 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785852 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmhns\" (UniqueName: \"kubernetes.io/projected/4449268e-1a20-4756-9d02-8939148e56ca-kube-api-access-vmhns\") pod \"network-check-source-869dcfcdcd-z95sx\" (UID: \"4449268e-1a20-4756-9d02-8939148e56ca\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-z95sx" Jul 16 09:09:20.785915 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785875 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fvgz9\" (UID: \"93e8fe28-842d-45a7-a34a-ccc2fa3ecf31\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" Jul 16 09:09:20.785915 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785910 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fa2a7c4-5499-408f-9c98-acc1debc7be0-tmp\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.786055 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.785978 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:20.786055 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.786021 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf55m\" (UniqueName: \"kubernetes.io/projected/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-kube-api-access-bf55m\") pod \"cluster-samples-operator-65db858894-fvgz9\" (UID: \"93e8fe28-842d-45a7-a34a-ccc2fa3ecf31\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" Jul 16 09:09:20.786055 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.786048 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fa2a7c4-5499-408f-9c98-acc1debc7be0-serving-cert\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.801080 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.801053 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmhns\" (UniqueName: \"kubernetes.io/projected/4449268e-1a20-4756-9d02-8939148e56ca-kube-api-access-vmhns\") pod \"network-check-source-869dcfcdcd-z95sx\" (UID: \"4449268e-1a20-4756-9d02-8939148e56ca\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-z95sx" Jul 16 09:09:20.801197 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.801142 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8pqh\" (UniqueName: \"kubernetes.io/projected/42c8464a-45ad-406d-b06a-cc2e8c6d1cbc-kube-api-access-v8pqh\") pod \"console-operator-7866497494-f4qh9\" (UID: \"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc\") " pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:20.887094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.886997 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4cf179b-d79d-44c6-a486-6cc872cc84ff-installation-pull-secrets\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.887094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887045 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcfe3047-1320-4d55-adb2-636de23660c4-serving-cert\") pod \"service-ca-operator-7695fc6c8c-kz47m\" (UID: \"dcfe3047-1320-4d55-adb2-636de23660c4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" Jul 16 09:09:20.887094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887067 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa2a7c4-5499-408f-9c98-acc1debc7be0-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.887094 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887089 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkp68\" (UniqueName: \"kubernetes.io/projected/dcfe3047-1320-4d55-adb2-636de23660c4-kube-api-access-bkp68\") pod \"service-ca-operator-7695fc6c8c-kz47m\" (UID: \"dcfe3047-1320-4d55-adb2-636de23660c4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" Jul 16 09:09:20.887402 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887113 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9fa2a7c4-5499-408f-9c98-acc1debc7be0-snapshots\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.887402 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887231 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4cf179b-d79d-44c6-a486-6cc872cc84ff-image-registry-private-configuration\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.887402 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887279 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-xqqv6\" (UID: \"7d93ba98-9730-4d8e-9ab5-d5111a98387c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" Jul 16 09:09:20.887402 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887306 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa2a7c4-5499-408f-9c98-acc1debc7be0-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.887402 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887333 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptck5\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-kube-api-access-ptck5\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.887402 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887364 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ws9t\" (UniqueName: \"kubernetes.io/projected/8322ecd6-180e-462a-97d2-8cd3942f208d-kube-api-access-2ws9t\") pod \"volume-data-source-validator-78495597f6-zfjdj\" (UID: \"8322ecd6-180e-462a-97d2-8cd3942f208d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-zfjdj" Jul 16 09:09:20.887402 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887401 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.887729 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:20.887414 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 09:09:20.887729 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887425 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dcfe3047-1320-4d55-adb2-636de23660c4-config\") pod \"service-ca-operator-7695fc6c8c-kz47m\" (UID: \"dcfe3047-1320-4d55-adb2-636de23660c4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" Jul 16 09:09:20.887729 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887459 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-certificates\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.887729 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:20.887491 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert podName:7d93ba98-9730-4d8e-9ab5-d5111a98387c nodeName:}" failed. No retries permitted until 2026-07-16 09:09:21.387469924 +0000 UTC m=+236.797018161 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-xqqv6" (UID: "7d93ba98-9730-4d8e-9ab5-d5111a98387c") : secret "networking-console-plugin-cert" not found Jul 16 09:09:20.887729 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887526 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d93ba98-9730-4d8e-9ab5-d5111a98387c-nginx-conf\") pod \"networking-console-plugin-6cf856758c-xqqv6\" (UID: \"7d93ba98-9730-4d8e-9ab5-d5111a98387c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" Jul 16 09:09:20.887729 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887570 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdsxt\" (UniqueName: \"kubernetes.io/projected/9fa2a7c4-5499-408f-9c98-acc1debc7be0-kube-api-access-bdsxt\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.887729 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887602 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/320437bb-7863-4d89-8736-4c69fe673ad5-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:20.888107 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887733 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa2a7c4-5499-408f-9c98-acc1debc7be0-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.888107 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887789 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9fa2a7c4-5499-408f-9c98-acc1debc7be0-snapshots\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.888107 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:20.887857 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 09:09:20.888107 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:20.887880 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5b8d48f6-r62kk: secret "image-registry-tls" not found Jul 16 09:09:20.888107 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887925 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl7dw\" (UniqueName: \"kubernetes.io/projected/320437bb-7863-4d89-8736-4c69fe673ad5-kube-api-access-hl7dw\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:20.888107 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:20.887947 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls podName:d4cf179b-d79d-44c6-a486-6cc872cc84ff nodeName:}" failed. No retries permitted until 2026-07-16 09:09:21.387926134 +0000 UTC m=+236.797474375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls") pod "image-registry-5b5b8d48f6-r62kk" (UID: "d4cf179b-d79d-44c6-a486-6cc872cc84ff") : secret "image-registry-tls" not found Jul 16 09:09:20.888107 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.887976 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4cf179b-d79d-44c6-a486-6cc872cc84ff-trusted-ca\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.888107 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.888006 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fa2a7c4-5499-408f-9c98-acc1debc7be0-tmp\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.888107 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.888038 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:20.888107 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.888059 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-certificates\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.888107 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.888073 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fa2a7c4-5499-408f-9c98-acc1debc7be0-serving-cert\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.888710 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.888124 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-bound-sa-token\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.888710 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.888160 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4cf179b-d79d-44c6-a486-6cc872cc84ff-ca-trust-extracted\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.888710 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.888377 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/320437bb-7863-4d89-8736-4c69fe673ad5-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:20.888710 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.888518 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4cf179b-d79d-44c6-a486-6cc872cc84ff-ca-trust-extracted\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.888710 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.888693 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dcfe3047-1320-4d55-adb2-636de23660c4-config\") pod \"service-ca-operator-7695fc6c8c-kz47m\" (UID: \"dcfe3047-1320-4d55-adb2-636de23660c4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" Jul 16 09:09:20.889042 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.889003 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fa2a7c4-5499-408f-9c98-acc1debc7be0-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.889117 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:20.889096 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 09:09:20.889173 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:20.889161 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls podName:320437bb-7863-4d89-8736-4c69fe673ad5 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:21.389144633 +0000 UTC m=+236.798692874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-vbs25" (UID: "320437bb-7863-4d89-8736-4c69fe673ad5") : secret "cluster-monitoring-operator-tls" not found Jul 16 09:09:20.889173 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.889161 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fa2a7c4-5499-408f-9c98-acc1debc7be0-tmp\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.889279 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.889259 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:20.889597 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.889572 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4cf179b-d79d-44c6-a486-6cc872cc84ff-trusted-ca\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.889760 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.889737 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d93ba98-9730-4d8e-9ab5-d5111a98387c-nginx-conf\") pod \"networking-console-plugin-6cf856758c-xqqv6\" (UID: \"7d93ba98-9730-4d8e-9ab5-d5111a98387c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" Jul 16 09:09:20.889973 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.889955 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4cf179b-d79d-44c6-a486-6cc872cc84ff-installation-pull-secrets\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.890155 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.890134 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcfe3047-1320-4d55-adb2-636de23660c4-serving-cert\") pod \"service-ca-operator-7695fc6c8c-kz47m\" (UID: \"dcfe3047-1320-4d55-adb2-636de23660c4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" Jul 16 09:09:20.890590 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.890570 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fa2a7c4-5499-408f-9c98-acc1debc7be0-serving-cert\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.891194 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.891169 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4cf179b-d79d-44c6-a486-6cc872cc84ff-image-registry-private-configuration\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.896566 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.896547 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-z95sx" Jul 16 09:09:20.899781 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.899756 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl7dw\" (UniqueName: \"kubernetes.io/projected/320437bb-7863-4d89-8736-4c69fe673ad5-kube-api-access-hl7dw\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:20.899909 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.899865 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ws9t\" (UniqueName: \"kubernetes.io/projected/8322ecd6-180e-462a-97d2-8cd3942f208d-kube-api-access-2ws9t\") pod \"volume-data-source-validator-78495597f6-zfjdj\" (UID: \"8322ecd6-180e-462a-97d2-8cd3942f208d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-zfjdj" Jul 16 09:09:20.900206 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.900180 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdsxt\" (UniqueName: \"kubernetes.io/projected/9fa2a7c4-5499-408f-9c98-acc1debc7be0-kube-api-access-bdsxt\") pod \"insights-operator-68d9dd94cf-5gbht\" (UID: \"9fa2a7c4-5499-408f-9c98-acc1debc7be0\") " pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:20.901459 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.901216 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkp68\" (UniqueName: \"kubernetes.io/projected/dcfe3047-1320-4d55-adb2-636de23660c4-kube-api-access-bkp68\") pod \"service-ca-operator-7695fc6c8c-kz47m\" (UID: \"dcfe3047-1320-4d55-adb2-636de23660c4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" Jul 16 09:09:20.901857 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.901643 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-bound-sa-token\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.901911 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.901885 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptck5\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-kube-api-access-ptck5\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:20.995230 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:20.995204 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" Jul 16 09:09:21.007685 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.007660 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" Jul 16 09:09:21.020264 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.020232 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-f4qh9"] Jul 16 09:09:21.020344 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.020269 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-zfjdj" Jul 16 09:09:21.022908 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:09:21.022884 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42c8464a_45ad_406d_b06a_cc2e8c6d1cbc.slice/crio-2a4d69e1b679d314d01f5bea1e4ce1beb891b0591cfcf2d1825b256958fe799a WatchSource:0}: Error finding container 2a4d69e1b679d314d01f5bea1e4ce1beb891b0591cfcf2d1825b256958fe799a: Status 404 returned error can't find the container with id 2a4d69e1b679d314d01f5bea1e4ce1beb891b0591cfcf2d1825b256958fe799a Jul 16 09:09:21.033141 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.033021 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-z95sx"] Jul 16 09:09:21.038280 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:09:21.038253 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4449268e_1a20_4756_9d02_8939148e56ca.slice/crio-c733a84a1460a500dca207c742aa961d7a3b27d321588b4b716b13391c2654b9 WatchSource:0}: Error finding container c733a84a1460a500dca207c742aa961d7a3b27d321588b4b716b13391c2654b9: Status 404 returned error can't find the container with id c733a84a1460a500dca207c742aa961d7a3b27d321588b4b716b13391c2654b9 Jul 16 09:09:21.148460 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.148431 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-5gbht"] Jul 16 09:09:21.151953 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:09:21.151926 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fa2a7c4_5499_408f_9c98_acc1debc7be0.slice/crio-921a251b670949185dcc7d39a2d828c008972211325171f92a0bbc7d1f14a5cc WatchSource:0}: Error finding container 921a251b670949185dcc7d39a2d828c008972211325171f92a0bbc7d1f14a5cc: Status 404 returned error can't find the container with id 921a251b670949185dcc7d39a2d828c008972211325171f92a0bbc7d1f14a5cc Jul 16 09:09:21.167383 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.167356 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m"] Jul 16 09:09:21.171336 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:09:21.171309 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcfe3047_1320_4d55_adb2_636de23660c4.slice/crio-e78f44f784ebab8a27ae8d4111679cec61010f25e246b1b69a028ee38e5cf183 WatchSource:0}: Error finding container e78f44f784ebab8a27ae8d4111679cec61010f25e246b1b69a028ee38e5cf183: Status 404 returned error can't find the container with id e78f44f784ebab8a27ae8d4111679cec61010f25e246b1b69a028ee38e5cf183 Jul 16 09:09:21.184341 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.184321 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-zfjdj"] Jul 16 09:09:21.186634 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:09:21.186609 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8322ecd6_180e_462a_97d2_8cd3942f208d.slice/crio-1891290af6a51a23f93041d895cee87a5d64bcee0eb9f0f0edb4ff3f747a51f6 WatchSource:0}: Error finding container 1891290af6a51a23f93041d895cee87a5d64bcee0eb9f0f0edb4ff3f747a51f6: Status 404 returned error can't find the container with id 1891290af6a51a23f93041d895cee87a5d64bcee0eb9f0f0edb4ff3f747a51f6 Jul 16 09:09:21.392250 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.392220 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:21.392416 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.392279 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-xqqv6\" (UID: \"7d93ba98-9730-4d8e-9ab5-d5111a98387c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" Jul 16 09:09:21.392416 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.392305 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:21.392416 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:21.392392 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 09:09:21.392587 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:21.392420 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 09:09:21.392587 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:21.392427 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 09:09:21.392587 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:21.392430 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5b8d48f6-r62kk: secret "image-registry-tls" not found Jul 16 09:09:21.392587 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:21.392462 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls podName:320437bb-7863-4d89-8736-4c69fe673ad5 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:22.392443731 +0000 UTC m=+237.801991977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-vbs25" (UID: "320437bb-7863-4d89-8736-4c69fe673ad5") : secret "cluster-monitoring-operator-tls" not found Jul 16 09:09:21.392587 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:21.392499 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert podName:7d93ba98-9730-4d8e-9ab5-d5111a98387c nodeName:}" failed. No retries permitted until 2026-07-16 09:09:22.392487 +0000 UTC m=+237.802035236 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-xqqv6" (UID: "7d93ba98-9730-4d8e-9ab5-d5111a98387c") : secret "networking-console-plugin-cert" not found Jul 16 09:09:21.392587 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:21.392511 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls podName:d4cf179b-d79d-44c6-a486-6cc872cc84ff nodeName:}" failed. No retries permitted until 2026-07-16 09:09:22.392505198 +0000 UTC m=+237.802053436 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls") pod "image-registry-5b5b8d48f6-r62kk" (UID: "d4cf179b-d79d-44c6-a486-6cc872cc84ff") : secret "image-registry-tls" not found Jul 16 09:09:21.452431 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.452330 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 16 09:09:21.529916 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.529891 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 16 09:09:21.536167 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:21.536149 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 09:09:21.536225 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:21.536213 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls podName:93e8fe28-842d-45a7-a34a-ccc2fa3ecf31 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:22.036192229 +0000 UTC m=+237.445740467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls") pod "cluster-samples-operator-65db858894-fvgz9" (UID: "93e8fe28-842d-45a7-a34a-ccc2fa3ecf31") : secret "samples-operator-tls" not found Jul 16 09:09:21.801950 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.801915 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-dqt7g\"" Jul 16 09:09:21.935437 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.935385 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" event={"ID":"dcfe3047-1320-4d55-adb2-636de23660c4","Type":"ContainerStarted","Data":"e78f44f784ebab8a27ae8d4111679cec61010f25e246b1b69a028ee38e5cf183"} Jul 16 09:09:21.937268 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.937240 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" event={"ID":"9fa2a7c4-5499-408f-9c98-acc1debc7be0","Type":"ContainerStarted","Data":"921a251b670949185dcc7d39a2d828c008972211325171f92a0bbc7d1f14a5cc"} Jul 16 09:09:21.938564 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.938501 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-zfjdj" event={"ID":"8322ecd6-180e-462a-97d2-8cd3942f208d","Type":"ContainerStarted","Data":"1891290af6a51a23f93041d895cee87a5d64bcee0eb9f0f0edb4ff3f747a51f6"} Jul 16 09:09:21.940950 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.940400 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-z95sx" event={"ID":"4449268e-1a20-4756-9d02-8939148e56ca","Type":"ContainerStarted","Data":"c0c1e046a41865c0ea6b7492a4716eaba5a2694ef1455e80c397d159ccdbb08a"} Jul 16 09:09:21.940950 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.940443 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-z95sx" event={"ID":"4449268e-1a20-4756-9d02-8939148e56ca","Type":"ContainerStarted","Data":"c733a84a1460a500dca207c742aa961d7a3b27d321588b4b716b13391c2654b9"} Jul 16 09:09:21.942666 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.942614 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-f4qh9" event={"ID":"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc","Type":"ContainerStarted","Data":"2a4d69e1b679d314d01f5bea1e4ce1beb891b0591cfcf2d1825b256958fe799a"} Jul 16 09:09:21.956181 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.956130 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-z95sx" podStartSLOduration=1.9561113159999999 podStartE2EDuration="1.956111316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:09:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:09:21.955338025 +0000 UTC m=+237.364886301" watchObservedRunningTime="2026-07-16 09:09:21.956111316 +0000 UTC m=+237.365659577" Jul 16 09:09:21.957465 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.957250 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 16 09:09:21.966183 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:21.966132 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf55m\" (UniqueName: \"kubernetes.io/projected/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-kube-api-access-bf55m\") pod \"cluster-samples-operator-65db858894-fvgz9\" (UID: \"93e8fe28-842d-45a7-a34a-ccc2fa3ecf31\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" Jul 16 09:09:22.098548 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:22.098464 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fvgz9\" (UID: \"93e8fe28-842d-45a7-a34a-ccc2fa3ecf31\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" Jul 16 09:09:22.098721 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:22.098625 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 09:09:22.098721 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:22.098683 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls podName:93e8fe28-842d-45a7-a34a-ccc2fa3ecf31 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:23.098666421 +0000 UTC m=+238.508214657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls") pod "cluster-samples-operator-65db858894-fvgz9" (UID: "93e8fe28-842d-45a7-a34a-ccc2fa3ecf31") : secret "samples-operator-tls" not found Jul 16 09:09:22.402071 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:22.401577 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-xqqv6\" (UID: \"7d93ba98-9730-4d8e-9ab5-d5111a98387c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" Jul 16 09:09:22.402071 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:22.401637 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:22.402071 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:22.401707 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:22.402071 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:22.402032 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 09:09:22.402344 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:22.402080 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 09:09:22.402344 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:22.402100 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5b8d48f6-r62kk: secret "image-registry-tls" not found Jul 16 09:09:22.402344 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:22.402107 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert podName:7d93ba98-9730-4d8e-9ab5-d5111a98387c nodeName:}" failed. No retries permitted until 2026-07-16 09:09:24.402085796 +0000 UTC m=+239.811634049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-xqqv6" (UID: "7d93ba98-9730-4d8e-9ab5-d5111a98387c") : secret "networking-console-plugin-cert" not found Jul 16 09:09:22.402344 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:22.402040 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 09:09:22.402344 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:22.402148 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls podName:d4cf179b-d79d-44c6-a486-6cc872cc84ff nodeName:}" failed. No retries permitted until 2026-07-16 09:09:24.402131996 +0000 UTC m=+239.811680238 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls") pod "image-registry-5b5b8d48f6-r62kk" (UID: "d4cf179b-d79d-44c6-a486-6cc872cc84ff") : secret "image-registry-tls" not found Jul 16 09:09:22.402344 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:22.402166 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls podName:320437bb-7863-4d89-8736-4c69fe673ad5 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:24.402156645 +0000 UTC m=+239.811704888 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-vbs25" (UID: "320437bb-7863-4d89-8736-4c69fe673ad5") : secret "cluster-monitoring-operator-tls" not found Jul 16 09:09:23.106766 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:23.106732 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fvgz9\" (UID: \"93e8fe28-842d-45a7-a34a-ccc2fa3ecf31\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" Jul 16 09:09:23.107246 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:23.106903 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 09:09:23.107246 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:23.106986 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls podName:93e8fe28-842d-45a7-a34a-ccc2fa3ecf31 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:25.10696586 +0000 UTC m=+240.516514098 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls") pod "cluster-samples-operator-65db858894-fvgz9" (UID: "93e8fe28-842d-45a7-a34a-ccc2fa3ecf31") : secret "samples-operator-tls" not found Jul 16 09:09:24.419302 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:24.419264 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:24.419731 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:24.419338 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-xqqv6\" (UID: \"7d93ba98-9730-4d8e-9ab5-d5111a98387c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" Jul 16 09:09:24.419731 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:24.419362 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:24.419731 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:24.419447 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 09:09:24.419731 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:24.419483 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 09:09:24.419731 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:24.419497 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5b8d48f6-r62kk: secret "image-registry-tls" not found Jul 16 09:09:24.419731 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:24.419494 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 09:09:24.419731 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:24.419530 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls podName:320437bb-7863-4d89-8736-4c69fe673ad5 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:28.419509086 +0000 UTC m=+243.829057327 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-vbs25" (UID: "320437bb-7863-4d89-8736-4c69fe673ad5") : secret "cluster-monitoring-operator-tls" not found Jul 16 09:09:24.419731 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:24.419553 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert podName:7d93ba98-9730-4d8e-9ab5-d5111a98387c nodeName:}" failed. No retries permitted until 2026-07-16 09:09:28.419538276 +0000 UTC m=+243.829086514 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-xqqv6" (UID: "7d93ba98-9730-4d8e-9ab5-d5111a98387c") : secret "networking-console-plugin-cert" not found Jul 16 09:09:24.419731 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:24.419568 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls podName:d4cf179b-d79d-44c6-a486-6cc872cc84ff nodeName:}" failed. No retries permitted until 2026-07-16 09:09:28.419560744 +0000 UTC m=+243.829108980 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls") pod "image-registry-5b5b8d48f6-r62kk" (UID: "d4cf179b-d79d-44c6-a486-6cc872cc84ff") : secret "image-registry-tls" not found Jul 16 09:09:24.953164 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:24.953074 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" event={"ID":"dcfe3047-1320-4d55-adb2-636de23660c4","Type":"ContainerStarted","Data":"bd0afbabbd2cb5f734280f3d957d8926671afb785f61c81da84af7e5254c2898"} Jul 16 09:09:24.954587 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:24.954560 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" event={"ID":"9fa2a7c4-5499-408f-9c98-acc1debc7be0","Type":"ContainerStarted","Data":"a2fff93f48211d82fd621d6da002a60d5cd4f86aed708f3bf2d5b3fdfda2bd81"} Jul 16 09:09:24.956055 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:24.956030 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-zfjdj" event={"ID":"8322ecd6-180e-462a-97d2-8cd3942f208d","Type":"ContainerStarted","Data":"bf63aaf652518f616527c45a86001166398d237aec89a1f7c82efc83f455872f"} Jul 16 09:09:24.957603 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:24.957583 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/0.log" Jul 16 09:09:24.957722 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:24.957624 2567 generic.go:356] "Generic (PLEG): container finished" podID="42c8464a-45ad-406d-b06a-cc2e8c6d1cbc" containerID="00ddd91a15275574091d1044016b3df9e82022af8c05926584904f22df9d7a91" exitCode=255 Jul 16 09:09:24.957722 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:24.957653 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-f4qh9" event={"ID":"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc","Type":"ContainerDied","Data":"00ddd91a15275574091d1044016b3df9e82022af8c05926584904f22df9d7a91"} Jul 16 09:09:24.957851 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:24.957837 2567 scope.go:117] "RemoveContainer" containerID="00ddd91a15275574091d1044016b3df9e82022af8c05926584904f22df9d7a91" Jul 16 09:09:24.975730 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:24.975185 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" podStartSLOduration=1.60828002 podStartE2EDuration="4.975166897s" totalImagesPullingTime="3.366886877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:09:24.969333351 +0000 UTC m=+240.378881608" watchObservedRunningTime="2026-07-16 09:09:24.975166897 +0000 UTC m=+240.384715159" Jul 16 09:09:24.993550 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:24.992641 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" podStartSLOduration=1.606197677 podStartE2EDuration="4.992620829s" totalImagesPullingTime="3.386423152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:09:24.992385665 +0000 UTC m=+240.401933944" watchObservedRunningTime="2026-07-16 09:09:24.992620829 +0000 UTC m=+240.402169091" Jul 16 09:09:25.009811 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:25.009771 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-zfjdj" podStartSLOduration=1.619980434 podStartE2EDuration="5.00975574s" totalImagesPullingTime="3.389775306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:09:25.009668589 +0000 UTC m=+240.419216849" watchObservedRunningTime="2026-07-16 09:09:25.00975574 +0000 UTC m=+240.419303991" Jul 16 09:09:25.125637 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:25.125602 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fvgz9\" (UID: \"93e8fe28-842d-45a7-a34a-ccc2fa3ecf31\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" Jul 16 09:09:25.125773 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:25.125754 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 09:09:25.125853 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:25.125842 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls podName:93e8fe28-842d-45a7-a34a-ccc2fa3ecf31 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:29.125810489 +0000 UTC m=+244.535358737 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls") pod "cluster-samples-operator-65db858894-fvgz9" (UID: "93e8fe28-842d-45a7-a34a-ccc2fa3ecf31") : secret "samples-operator-tls" not found Jul 16 09:09:25.962494 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:25.962415 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:09:25.962905 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:25.962865 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/0.log" Jul 16 09:09:25.962965 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:25.962901 2567 generic.go:356] "Generic (PLEG): container finished" podID="42c8464a-45ad-406d-b06a-cc2e8c6d1cbc" containerID="ea41d3a871ce0fe1b7efda82626bc87ee46d5ac9046d6722213b63c179b121e5" exitCode=255 Jul 16 09:09:25.963050 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:25.963015 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-f4qh9" event={"ID":"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc","Type":"ContainerDied","Data":"ea41d3a871ce0fe1b7efda82626bc87ee46d5ac9046d6722213b63c179b121e5"} Jul 16 09:09:25.963132 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:25.963118 2567 scope.go:117] "RemoveContainer" containerID="00ddd91a15275574091d1044016b3df9e82022af8c05926584904f22df9d7a91" Jul 16 09:09:25.963186 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:25.963169 2567 scope.go:117] "RemoveContainer" containerID="ea41d3a871ce0fe1b7efda82626bc87ee46d5ac9046d6722213b63c179b121e5" Jul 16 09:09:25.963430 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:25.963392 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-f4qh9_openshift-console-operator(42c8464a-45ad-406d-b06a-cc2e8c6d1cbc)\"" pod="openshift-console-operator/console-operator-7866497494-f4qh9" podUID="42c8464a-45ad-406d-b06a-cc2e8c6d1cbc" Jul 16 09:09:26.155789 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.155757 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mm9f5"] Jul 16 09:09:26.158975 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.158948 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.161929 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.161910 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 09:09:26.162301 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.162283 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gw7j9\"" Jul 16 09:09:26.162353 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.162311 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 09:09:26.180290 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.180261 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mm9f5"] Jul 16 09:09:26.235699 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.235630 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4323d88c-6520-4845-816b-aeb12c4afc85-data-volume\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.235699 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.235686 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4323d88c-6520-4845-816b-aeb12c4afc85-crio-socket\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.235893 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.235753 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.235893 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.235781 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42lzs\" (UniqueName: \"kubernetes.io/projected/4323d88c-6520-4845-816b-aeb12c4afc85-kube-api-access-42lzs\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.235893 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.235836 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.336535 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.336501 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.336535 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.336535 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42lzs\" (UniqueName: \"kubernetes.io/projected/4323d88c-6520-4845-816b-aeb12c4afc85-kube-api-access-42lzs\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.336745 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.336601 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.336745 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.336647 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4323d88c-6520-4845-816b-aeb12c4afc85-data-volume\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.336745 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:26.336651 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 09:09:26.336745 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.336680 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4323d88c-6520-4845-816b-aeb12c4afc85-crio-socket\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.336745 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:26.336719 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls podName:4323d88c-6520-4845-816b-aeb12c4afc85 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:26.836703498 +0000 UTC m=+242.246251735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mm9f5" (UID: "4323d88c-6520-4845-816b-aeb12c4afc85") : secret "insights-runtime-extractor-tls" not found Jul 16 09:09:26.336970 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.336755 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4323d88c-6520-4845-816b-aeb12c4afc85-crio-socket\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.336970 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:26.336759 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 09:09:26.336970 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:26.336847 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm podName:4323d88c-6520-4845-816b-aeb12c4afc85 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:26.836834744 +0000 UTC m=+242.246382986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mm9f5" (UID: "4323d88c-6520-4845-816b-aeb12c4afc85") : configmap "kube-rbac-proxy" not found Jul 16 09:09:26.337077 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.337059 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4323d88c-6520-4845-816b-aeb12c4afc85-data-volume\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.351766 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.351746 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42lzs\" (UniqueName: \"kubernetes.io/projected/4323d88c-6520-4845-816b-aeb12c4afc85-kube-api-access-42lzs\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.420362 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.420321 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-94bkn"] Jul 16 09:09:26.422622 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.422604 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-94bkn" Jul 16 09:09:26.426605 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.426580 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-t2v9x\"" Jul 16 09:09:26.426948 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.426934 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 16 09:09:26.427061 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.427046 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 16 09:09:26.433854 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.433831 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-94bkn"] Jul 16 09:09:26.538692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.538657 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgxnw\" (UniqueName: \"kubernetes.io/projected/0aaa2820-a527-477c-a6f0-e95b97f14f5a-kube-api-access-jgxnw\") pod \"migrator-7f59b54f6c-94bkn\" (UID: \"0aaa2820-a527-477c-a6f0-e95b97f14f5a\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-94bkn" Jul 16 09:09:26.639971 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.639930 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgxnw\" (UniqueName: \"kubernetes.io/projected/0aaa2820-a527-477c-a6f0-e95b97f14f5a-kube-api-access-jgxnw\") pod \"migrator-7f59b54f6c-94bkn\" (UID: \"0aaa2820-a527-477c-a6f0-e95b97f14f5a\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-94bkn" Jul 16 09:09:26.649175 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.649151 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgxnw\" (UniqueName: \"kubernetes.io/projected/0aaa2820-a527-477c-a6f0-e95b97f14f5a-kube-api-access-jgxnw\") pod \"migrator-7f59b54f6c-94bkn\" (UID: \"0aaa2820-a527-477c-a6f0-e95b97f14f5a\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-94bkn" Jul 16 09:09:26.731741 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.731707 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-94bkn" Jul 16 09:09:26.841597 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.841569 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.841726 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.841613 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:26.841726 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:26.841686 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 09:09:26.841804 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:26.841720 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 09:09:26.841804 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:26.841736 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm podName:4323d88c-6520-4845-816b-aeb12c4afc85 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:27.841722501 +0000 UTC m=+243.251270738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mm9f5" (UID: "4323d88c-6520-4845-816b-aeb12c4afc85") : configmap "kube-rbac-proxy" not found Jul 16 09:09:26.841804 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:26.841774 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls podName:4323d88c-6520-4845-816b-aeb12c4afc85 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:27.841760649 +0000 UTC m=+243.251308885 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mm9f5" (UID: "4323d88c-6520-4845-816b-aeb12c4afc85") : secret "insights-runtime-extractor-tls" not found Jul 16 09:09:26.849132 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.849103 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-94bkn"] Jul 16 09:09:26.851593 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:09:26.851568 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aaa2820_a527_477c_a6f0_e95b97f14f5a.slice/crio-7a1a0993fbccc74367ec6d91373fa835c7edad21c1f1e9940aa979a1dd2f6263 WatchSource:0}: Error finding container 7a1a0993fbccc74367ec6d91373fa835c7edad21c1f1e9940aa979a1dd2f6263: Status 404 returned error can't find the container with id 7a1a0993fbccc74367ec6d91373fa835c7edad21c1f1e9940aa979a1dd2f6263 Jul 16 09:09:26.967646 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.967620 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:09:26.968053 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.967993 2567 scope.go:117] "RemoveContainer" containerID="ea41d3a871ce0fe1b7efda82626bc87ee46d5ac9046d6722213b63c179b121e5" Jul 16 09:09:26.968170 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:26.968152 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-f4qh9_openshift-console-operator(42c8464a-45ad-406d-b06a-cc2e8c6d1cbc)\"" pod="openshift-console-operator/console-operator-7866497494-f4qh9" podUID="42c8464a-45ad-406d-b06a-cc2e8c6d1cbc" Jul 16 09:09:26.968928 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:26.968898 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-94bkn" event={"ID":"0aaa2820-a527-477c-a6f0-e95b97f14f5a","Type":"ContainerStarted","Data":"7a1a0993fbccc74367ec6d91373fa835c7edad21c1f1e9940aa979a1dd2f6263"} Jul 16 09:09:27.851171 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:27.851135 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:27.851358 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:27.851212 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:27.851358 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:27.851295 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 09:09:27.851358 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:27.851334 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 09:09:27.851519 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:27.851374 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls podName:4323d88c-6520-4845-816b-aeb12c4afc85 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:29.851354082 +0000 UTC m=+245.260902330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mm9f5" (UID: "4323d88c-6520-4845-816b-aeb12c4afc85") : secret "insights-runtime-extractor-tls" not found Jul 16 09:09:27.851519 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:27.851395 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm podName:4323d88c-6520-4845-816b-aeb12c4afc85 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:29.851385623 +0000 UTC m=+245.260933861 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mm9f5" (UID: "4323d88c-6520-4845-816b-aeb12c4afc85") : configmap "kube-rbac-proxy" not found Jul 16 09:09:27.921724 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:27.921699 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-78b27_850750a5-782b-490b-b29a-ae2e05d1aff8/dns-node-resolver/0.log" Jul 16 09:09:28.456645 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.456606 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:28.457066 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.456673 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-xqqv6\" (UID: \"7d93ba98-9730-4d8e-9ab5-d5111a98387c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" Jul 16 09:09:28.457066 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.456696 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:28.457066 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:28.456765 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 09:09:28.457066 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:28.456813 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 09:09:28.457066 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:28.456859 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls podName:320437bb-7863-4d89-8736-4c69fe673ad5 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:36.45684211 +0000 UTC m=+251.866390348 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-vbs25" (UID: "320437bb-7863-4d89-8736-4c69fe673ad5") : secret "cluster-monitoring-operator-tls" not found Jul 16 09:09:28.457066 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:28.456775 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 09:09:28.457066 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:28.456886 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5b8d48f6-r62kk: secret "image-registry-tls" not found Jul 16 09:09:28.457066 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:28.456902 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert podName:7d93ba98-9730-4d8e-9ab5-d5111a98387c nodeName:}" failed. No retries permitted until 2026-07-16 09:09:36.456887086 +0000 UTC m=+251.866435338 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-xqqv6" (UID: "7d93ba98-9730-4d8e-9ab5-d5111a98387c") : secret "networking-console-plugin-cert" not found Jul 16 09:09:28.457066 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:28.456938 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls podName:d4cf179b-d79d-44c6-a486-6cc872cc84ff nodeName:}" failed. No retries permitted until 2026-07-16 09:09:36.456920695 +0000 UTC m=+251.866468951 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls") pod "image-registry-5b5b8d48f6-r62kk" (UID: "d4cf179b-d79d-44c6-a486-6cc872cc84ff") : secret "image-registry-tls" not found Jul 16 09:09:28.508869 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.508795 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-8p7bp"] Jul 16 09:09:28.510929 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.510914 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-8p7bp" Jul 16 09:09:28.513591 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.513566 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 16 09:09:28.513752 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.513711 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 16 09:09:28.513844 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.513794 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 16 09:09:28.513844 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.513719 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 16 09:09:28.514686 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.514668 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-fdvsh\"" Jul 16 09:09:28.519705 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.519687 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-8p7bp"] Jul 16 09:09:28.557657 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.557630 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a8c5908c-a9e9-4f00-9c9c-2c912ff24260-signing-key\") pod \"service-ca-79f68b9f4f-8p7bp\" (UID: \"a8c5908c-a9e9-4f00-9c9c-2c912ff24260\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8p7bp" Jul 16 09:09:28.557795 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.557668 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a8c5908c-a9e9-4f00-9c9c-2c912ff24260-signing-cabundle\") pod \"service-ca-79f68b9f4f-8p7bp\" (UID: \"a8c5908c-a9e9-4f00-9c9c-2c912ff24260\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8p7bp" Jul 16 09:09:28.557795 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.557720 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w85t7\" (UniqueName: \"kubernetes.io/projected/a8c5908c-a9e9-4f00-9c9c-2c912ff24260-kube-api-access-w85t7\") pod \"service-ca-79f68b9f4f-8p7bp\" (UID: \"a8c5908c-a9e9-4f00-9c9c-2c912ff24260\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8p7bp" Jul 16 09:09:28.658192 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.658161 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a8c5908c-a9e9-4f00-9c9c-2c912ff24260-signing-key\") pod \"service-ca-79f68b9f4f-8p7bp\" (UID: \"a8c5908c-a9e9-4f00-9c9c-2c912ff24260\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8p7bp" Jul 16 09:09:28.658377 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.658203 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a8c5908c-a9e9-4f00-9c9c-2c912ff24260-signing-cabundle\") pod \"service-ca-79f68b9f4f-8p7bp\" (UID: \"a8c5908c-a9e9-4f00-9c9c-2c912ff24260\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8p7bp" Jul 16 09:09:28.658377 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.658230 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w85t7\" (UniqueName: \"kubernetes.io/projected/a8c5908c-a9e9-4f00-9c9c-2c912ff24260-kube-api-access-w85t7\") pod \"service-ca-79f68b9f4f-8p7bp\" (UID: \"a8c5908c-a9e9-4f00-9c9c-2c912ff24260\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8p7bp" Jul 16 09:09:28.658934 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.658906 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a8c5908c-a9e9-4f00-9c9c-2c912ff24260-signing-cabundle\") pod \"service-ca-79f68b9f4f-8p7bp\" (UID: \"a8c5908c-a9e9-4f00-9c9c-2c912ff24260\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8p7bp" Jul 16 09:09:28.660650 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.660629 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a8c5908c-a9e9-4f00-9c9c-2c912ff24260-signing-key\") pod \"service-ca-79f68b9f4f-8p7bp\" (UID: \"a8c5908c-a9e9-4f00-9c9c-2c912ff24260\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8p7bp" Jul 16 09:09:28.668617 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.668595 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w85t7\" (UniqueName: \"kubernetes.io/projected/a8c5908c-a9e9-4f00-9c9c-2c912ff24260-kube-api-access-w85t7\") pod \"service-ca-79f68b9f4f-8p7bp\" (UID: \"a8c5908c-a9e9-4f00-9c9c-2c912ff24260\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8p7bp" Jul 16 09:09:28.819929 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.819894 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-8p7bp" Jul 16 09:09:28.939897 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.939865 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-8p7bp"] Jul 16 09:09:28.944361 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:09:28.944331 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c5908c_a9e9_4f00_9c9c_2c912ff24260.slice/crio-669a6e993d86a8d2d736d978146863791a1594f3923148649a02ae7cc0c6ac53 WatchSource:0}: Error finding container 669a6e993d86a8d2d736d978146863791a1594f3923148649a02ae7cc0c6ac53: Status 404 returned error can't find the container with id 669a6e993d86a8d2d736d978146863791a1594f3923148649a02ae7cc0c6ac53 Jul 16 09:09:28.978526 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.978499 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-8p7bp" event={"ID":"a8c5908c-a9e9-4f00-9c9c-2c912ff24260","Type":"ContainerStarted","Data":"669a6e993d86a8d2d736d978146863791a1594f3923148649a02ae7cc0c6ac53"} Jul 16 09:09:28.980353 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.980322 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-94bkn" event={"ID":"0aaa2820-a527-477c-a6f0-e95b97f14f5a","Type":"ContainerStarted","Data":"6cfe1f53dfb8879f65c705531ce034a97bf348e22c2e0b5bf364e92206a90895"} Jul 16 09:09:28.980430 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.980379 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-94bkn" event={"ID":"0aaa2820-a527-477c-a6f0-e95b97f14f5a","Type":"ContainerStarted","Data":"dbb8890de8b9404d667dd5f5633abd446b1daa79bc8f8fca7ee0ad7964fcb28f"} Jul 16 09:09:28.998295 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:28.998256 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-94bkn" podStartSLOduration=1.6293361100000001 podStartE2EDuration="2.998243342s" totalImagesPullingTime="1.368907232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:09:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:09:28.997995204 +0000 UTC m=+244.407543489" watchObservedRunningTime="2026-07-16 09:09:28.998243342 +0000 UTC m=+244.407791600" Jul 16 09:09:29.131182 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:29.131111 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mprh4_3f14599c-3b35-405b-a145-1a861e050d1c/node-ca/0.log" Jul 16 09:09:29.162619 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:29.162586 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fvgz9\" (UID: \"93e8fe28-842d-45a7-a34a-ccc2fa3ecf31\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" Jul 16 09:09:29.162776 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:29.162760 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 09:09:29.162862 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:29.162850 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls podName:93e8fe28-842d-45a7-a34a-ccc2fa3ecf31 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:37.16281083 +0000 UTC m=+252.572359070 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls") pod "cluster-samples-operator-65db858894-fvgz9" (UID: "93e8fe28-842d-45a7-a34a-ccc2fa3ecf31") : secret "samples-operator-tls" not found Jul 16 09:09:29.721644 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:29.721615 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kzpnm_d2d206a0-1603-4fd8-936a-dacba9979e0c/serve-healthcheck-canary/0.log" Jul 16 09:09:29.868560 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:29.868524 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:29.868764 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:29.868580 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:29.868764 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:29.868666 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 09:09:29.868764 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:29.868692 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 09:09:29.868764 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:29.868744 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls podName:4323d88c-6520-4845-816b-aeb12c4afc85 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:33.86872627 +0000 UTC m=+249.278274507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mm9f5" (UID: "4323d88c-6520-4845-816b-aeb12c4afc85") : secret "insights-runtime-extractor-tls" not found Jul 16 09:09:29.868764 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:29.868761 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm podName:4323d88c-6520-4845-816b-aeb12c4afc85 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:33.868753154 +0000 UTC m=+249.278301393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mm9f5" (UID: "4323d88c-6520-4845-816b-aeb12c4afc85") : configmap "kube-rbac-proxy" not found Jul 16 09:09:29.985151 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:29.985063 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-8p7bp" event={"ID":"a8c5908c-a9e9-4f00-9c9c-2c912ff24260","Type":"ContainerStarted","Data":"63cc5cfd65f6fa3ddb3dfebd96ecf414118fb00c5ba3e3761c824b8e48533b9c"} Jul 16 09:09:30.002661 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:30.002619 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-8p7bp" podStartSLOduration=2.002602621 podStartE2EDuration="2.002602621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:09:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:09:30.000898922 +0000 UTC m=+245.410447202" watchObservedRunningTime="2026-07-16 09:09:30.002602621 +0000 UTC m=+245.412150928" Jul 16 09:09:30.889926 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:30.889840 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:30.889926 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:30.889906 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:30.890343 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:30.890330 2567 scope.go:117] "RemoveContainer" containerID="ea41d3a871ce0fe1b7efda82626bc87ee46d5ac9046d6722213b63c179b121e5" Jul 16 09:09:30.890524 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:30.890505 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-f4qh9_openshift-console-operator(42c8464a-45ad-406d-b06a-cc2e8c6d1cbc)\"" pod="openshift-console-operator/console-operator-7866497494-f4qh9" podUID="42c8464a-45ad-406d-b06a-cc2e8c6d1cbc" Jul 16 09:09:33.905141 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:33.905102 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:33.905526 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:33.905158 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:33.905526 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:33.905234 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 09:09:33.905526 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:33.905244 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 09:09:33.905526 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:33.905293 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm podName:4323d88c-6520-4845-816b-aeb12c4afc85 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:41.905278054 +0000 UTC m=+257.314826314 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mm9f5" (UID: "4323d88c-6520-4845-816b-aeb12c4afc85") : configmap "kube-rbac-proxy" not found Jul 16 09:09:33.905526 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:33.905307 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls podName:4323d88c-6520-4845-816b-aeb12c4afc85 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:41.905301047 +0000 UTC m=+257.314849284 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mm9f5" (UID: "4323d88c-6520-4845-816b-aeb12c4afc85") : secret "insights-runtime-extractor-tls" not found Jul 16 09:09:36.025751 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:36.025712 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:09:36.026172 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:36.025853 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 09:09:36.026172 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:36.025905 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs podName:9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd nodeName:}" failed. No retries permitted until 2026-07-16 09:11:38.025891084 +0000 UTC m=+373.435439321 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs") pod "network-metrics-daemon-ph7bq" (UID: "9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd") : secret "metrics-daemon-secret" not found Jul 16 09:09:36.530785 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:36.530752 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:36.530982 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:36.530848 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-xqqv6\" (UID: \"7d93ba98-9730-4d8e-9ab5-d5111a98387c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" Jul 16 09:09:36.530982 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:36.530879 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:36.530982 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:36.530924 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 09:09:36.531139 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:36.530984 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls podName:320437bb-7863-4d89-8736-4c69fe673ad5 nodeName:}" failed. No retries permitted until 2026-07-16 09:09:52.530970098 +0000 UTC m=+267.940518336 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-vbs25" (UID: "320437bb-7863-4d89-8736-4c69fe673ad5") : secret "cluster-monitoring-operator-tls" not found Jul 16 09:09:36.531139 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:36.531003 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 16 09:09:36.531139 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:09:36.531063 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert podName:7d93ba98-9730-4d8e-9ab5-d5111a98387c nodeName:}" failed. No retries permitted until 2026-07-16 09:09:52.531047261 +0000 UTC m=+267.940595512 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-xqqv6" (UID: "7d93ba98-9730-4d8e-9ab5-d5111a98387c") : secret "networking-console-plugin-cert" not found Jul 16 09:09:36.533200 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:36.533182 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls\") pod \"image-registry-5b5b8d48f6-r62kk\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:36.618028 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:36.617997 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8dsdg\"" Jul 16 09:09:36.626161 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:36.626137 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:36.757650 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:36.757605 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b5b8d48f6-r62kk"] Jul 16 09:09:36.760368 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:09:36.760321 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4cf179b_d79d_44c6_a486_6cc872cc84ff.slice/crio-381bf294741a4a1e20393b643915168d94e8848dcace99fad511466cef6b2ee1 WatchSource:0}: Error finding container 381bf294741a4a1e20393b643915168d94e8848dcace99fad511466cef6b2ee1: Status 404 returned error can't find the container with id 381bf294741a4a1e20393b643915168d94e8848dcace99fad511466cef6b2ee1 Jul 16 09:09:37.010200 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:37.010164 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" event={"ID":"d4cf179b-d79d-44c6-a486-6cc872cc84ff","Type":"ContainerStarted","Data":"ed2dbd4f2f9f4f2325f6daaf94cf8088ecc09e667688372bc504cde1abc31420"} Jul 16 09:09:37.010366 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:37.010223 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" event={"ID":"d4cf179b-d79d-44c6-a486-6cc872cc84ff","Type":"ContainerStarted","Data":"381bf294741a4a1e20393b643915168d94e8848dcace99fad511466cef6b2ee1"} Jul 16 09:09:37.010366 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:37.010259 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:09:37.030709 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:37.030663 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" podStartSLOduration=17.030646872 podStartE2EDuration="17.030646872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:09:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:09:37.030066738 +0000 UTC m=+252.439614987" watchObservedRunningTime="2026-07-16 09:09:37.030646872 +0000 UTC m=+252.440195124" Jul 16 09:09:37.239693 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:37.239598 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fvgz9\" (UID: \"93e8fe28-842d-45a7-a34a-ccc2fa3ecf31\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" Jul 16 09:09:37.241929 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:37.241901 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/93e8fe28-842d-45a7-a34a-ccc2fa3ecf31-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fvgz9\" (UID: \"93e8fe28-842d-45a7-a34a-ccc2fa3ecf31\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" Jul 16 09:09:37.405089 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:37.405060 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-dqt7g\"" Jul 16 09:09:37.413019 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:37.412989 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" Jul 16 09:09:37.562063 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:37.562026 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9"] Jul 16 09:09:38.014333 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:38.014289 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" event={"ID":"93e8fe28-842d-45a7-a34a-ccc2fa3ecf31","Type":"ContainerStarted","Data":"5ab650b9be1d71bb36316f8d572918164cd9262bf3a4ea8dc076404d239e550c"} Jul 16 09:09:40.022670 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:40.022633 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" event={"ID":"93e8fe28-842d-45a7-a34a-ccc2fa3ecf31","Type":"ContainerStarted","Data":"09a2c0b79694eb52f4ab21bbedf18846009971ec88f823e655460e1fdae151a5"} Jul 16 09:09:40.023095 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:40.022688 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" event={"ID":"93e8fe28-842d-45a7-a34a-ccc2fa3ecf31","Type":"ContainerStarted","Data":"f7625f695be2ac485e58bfb7c949ebba715e8b8d3abddd04c816f069ca7cc136"} Jul 16 09:09:40.043093 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:40.043045 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fvgz9" podStartSLOduration=18.067412866 podStartE2EDuration="20.043029127s" totalImagesPullingTime="1.975616261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:09:40.042353196 +0000 UTC m=+255.451901456" watchObservedRunningTime="2026-07-16 09:09:40.043029127 +0000 UTC m=+255.452577386" Jul 16 09:09:41.978888 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:41.978852 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:41.979262 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:41.978904 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:41.979486 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:41.979465 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4323d88c-6520-4845-816b-aeb12c4afc85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:41.981210 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:41.981189 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4323d88c-6520-4845-816b-aeb12c4afc85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mm9f5\" (UID: \"4323d88c-6520-4845-816b-aeb12c4afc85\") " pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:42.067957 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:42.067917 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mm9f5" Jul 16 09:09:42.193468 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:42.193417 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mm9f5"] Jul 16 09:09:42.195670 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:09:42.195646 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4323d88c_6520_4845_816b_aeb12c4afc85.slice/crio-acf23d91890f0d6ca3ac639a3674b0a4d32df408e56872c89f629c168890d215 WatchSource:0}: Error finding container acf23d91890f0d6ca3ac639a3674b0a4d32df408e56872c89f629c168890d215: Status 404 returned error can't find the container with id acf23d91890f0d6ca3ac639a3674b0a4d32df408e56872c89f629c168890d215 Jul 16 09:09:43.035682 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:43.035632 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mm9f5" event={"ID":"4323d88c-6520-4845-816b-aeb12c4afc85","Type":"ContainerStarted","Data":"acf23d91890f0d6ca3ac639a3674b0a4d32df408e56872c89f629c168890d215"} Jul 16 09:09:44.318888 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:44.318854 2567 scope.go:117] "RemoveContainer" containerID="ea41d3a871ce0fe1b7efda82626bc87ee46d5ac9046d6722213b63c179b121e5" Jul 16 09:09:47.052498 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:47.052466 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mm9f5" event={"ID":"4323d88c-6520-4845-816b-aeb12c4afc85","Type":"ContainerStarted","Data":"47c1f2266aa6a665a3016cb870ee1783cc1003b0b01673c769399ff30ed092a2"} Jul 16 09:09:47.054279 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:47.054258 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:09:47.054375 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:47.054325 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-f4qh9" event={"ID":"42c8464a-45ad-406d-b06a-cc2e8c6d1cbc","Type":"ContainerStarted","Data":"8c3b112305de83513ec8534bfb166844d517fb83bd0058b2166bbf1c0beab871"} Jul 16 09:09:47.054595 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:47.054564 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:47.071409 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:47.071370 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-f4qh9" podStartSLOduration=23.561005733000002 podStartE2EDuration="27.071356812s" totalImagesPullingTime="3.510351079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:09:47.070971292 +0000 UTC m=+262.480519551" watchObservedRunningTime="2026-07-16 09:09:47.071356812 +0000 UTC m=+262.480905071" Jul 16 09:09:47.500653 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:47.500618 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-f4qh9" Jul 16 09:09:51.071352 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:51.071320 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mm9f5" event={"ID":"4323d88c-6520-4845-816b-aeb12c4afc85","Type":"ContainerStarted","Data":"b932e4fb4ab91297a69881e7279f5623ca926d9ba62beb735ca2c8751c976c45"} Jul 16 09:09:51.901838 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:51.901783 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b5b8d48f6-r62kk"] Jul 16 09:09:51.907250 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:51.907136 2567 patch_prober.go:28] interesting pod/image-registry-5b5b8d48f6-r62kk container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 09:09:51.907250 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:51.907202 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" podUID="d4cf179b-d79d-44c6-a486-6cc872cc84ff" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 09:09:52.579508 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:52.579466 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:52.580012 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:52.579561 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-xqqv6\" (UID: \"7d93ba98-9730-4d8e-9ab5-d5111a98387c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" Jul 16 09:09:52.582235 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:52.582205 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/320437bb-7863-4d89-8736-4c69fe673ad5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-vbs25\" (UID: \"320437bb-7863-4d89-8736-4c69fe673ad5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:52.582359 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:52.582259 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d93ba98-9730-4d8e-9ab5-d5111a98387c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-xqqv6\" (UID: \"7d93ba98-9730-4d8e-9ab5-d5111a98387c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" Jul 16 09:09:52.711252 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:52.711223 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6cz65\"" Jul 16 09:09:52.716556 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:52.716533 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-m6lv5\"" Jul 16 09:09:52.718414 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:52.718389 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" Jul 16 09:09:52.725205 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:52.725185 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" Jul 16 09:09:54.202852 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:54.202809 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-xqqv6"] Jul 16 09:09:54.205740 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:09:54.205713 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d93ba98_9730_4d8e_9ab5_d5111a98387c.slice/crio-ab15cf92dc4bc378dac87ffa78ded89318491d736d9dfe502ee7a6300c1800d1 WatchSource:0}: Error finding container ab15cf92dc4bc378dac87ffa78ded89318491d736d9dfe502ee7a6300c1800d1: Status 404 returned error can't find the container with id ab15cf92dc4bc378dac87ffa78ded89318491d736d9dfe502ee7a6300c1800d1 Jul 16 09:09:54.215548 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:54.215527 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25"] Jul 16 09:09:54.218249 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:09:54.218226 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod320437bb_7863_4d89_8736_4c69fe673ad5.slice/crio-6df83e6a12e74bc86e543daa3beb21fde3c1f17490665be87f2f67d05d290012 WatchSource:0}: Error finding container 6df83e6a12e74bc86e543daa3beb21fde3c1f17490665be87f2f67d05d290012: Status 404 returned error can't find the container with id 6df83e6a12e74bc86e543daa3beb21fde3c1f17490665be87f2f67d05d290012 Jul 16 09:09:55.088357 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:55.088298 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mm9f5" event={"ID":"4323d88c-6520-4845-816b-aeb12c4afc85","Type":"ContainerStarted","Data":"58ecd634177fb81cd59de695a747515945247bf4c5ff220c0ab5b21b728f1efc"} Jul 16 09:09:55.089608 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:55.089578 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" event={"ID":"7d93ba98-9730-4d8e-9ab5-d5111a98387c","Type":"ContainerStarted","Data":"ab15cf92dc4bc378dac87ffa78ded89318491d736d9dfe502ee7a6300c1800d1"} Jul 16 09:09:55.090868 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:55.090841 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" event={"ID":"320437bb-7863-4d89-8736-4c69fe673ad5","Type":"ContainerStarted","Data":"6df83e6a12e74bc86e543daa3beb21fde3c1f17490665be87f2f67d05d290012"} Jul 16 09:09:55.109979 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:55.109930 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mm9f5" podStartSLOduration=17.297896837 podStartE2EDuration="29.109916695s" totalImagesPullingTime="11.812019858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 09:09:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:09:55.107912964 +0000 UTC m=+270.517461245" watchObservedRunningTime="2026-07-16 09:09:55.109916695 +0000 UTC m=+270.519464933" Jul 16 09:09:56.097245 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:56.097211 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" event={"ID":"7d93ba98-9730-4d8e-9ab5-d5111a98387c","Type":"ContainerStarted","Data":"b2cd40553e9485b583db3227832b02ec7b76dcfb6d0fb9c27f35f898e0e2d12e"} Jul 16 09:09:56.116938 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:56.116890 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-xqqv6" podStartSLOduration=35.017391799 podStartE2EDuration="36.116874999s" totalImagesPullingTime="1.0994832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:09:56.116047297 +0000 UTC m=+271.525595578" watchObservedRunningTime="2026-07-16 09:09:56.116874999 +0000 UTC m=+271.526423270" Jul 16 09:09:58.106420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:09:58.106387 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" event={"ID":"320437bb-7863-4d89-8736-4c69fe673ad5","Type":"ContainerStarted","Data":"3e2d332b00576a705ea4e02c950f60ee3a2a6ce3ce55bd5f069d1ff214ffc779"} Jul 16 09:10:01.906484 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:01.906449 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:10:01.928497 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:01.928449 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-vbs25" podStartSLOduration=39.134046294 podStartE2EDuration="41.928434674s" totalImagesPullingTime="2.79438838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:09:58.125150962 +0000 UTC m=+273.534699223" watchObservedRunningTime="2026-07-16 09:10:01.928434674 +0000 UTC m=+277.337982932" Jul 16 09:10:06.053944 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.053912 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7996v"] Jul 16 09:10:06.056795 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.056773 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.062234 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.062213 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 09:10:06.062374 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.062250 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tg9vk\"" Jul 16 09:10:06.063279 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.063261 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 09:10:06.063377 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.063306 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 09:10:06.063721 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.063707 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 09:10:06.084498 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.084466 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.084612 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.084512 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b83a288-4615-4ee7-a2d4-fb0475223943-metrics-client-ca\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.084612 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.084566 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgxx6\" (UniqueName: \"kubernetes.io/projected/8b83a288-4615-4ee7-a2d4-fb0475223943-kube-api-access-sgxx6\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.084727 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.084624 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-tls\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.084727 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.084651 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-accelerators-collector-config\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.084727 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.084706 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-textfile\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.084885 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.084736 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b83a288-4615-4ee7-a2d4-fb0475223943-sys\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.084885 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.084778 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b83a288-4615-4ee7-a2d4-fb0475223943-root\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.084885 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.084812 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-wtmp\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.185385 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.185342 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.185644 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.185628 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b83a288-4615-4ee7-a2d4-fb0475223943-metrics-client-ca\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.186374 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.186345 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgxx6\" (UniqueName: \"kubernetes.io/projected/8b83a288-4615-4ee7-a2d4-fb0475223943-kube-api-access-sgxx6\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.186867 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.186283 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b83a288-4615-4ee7-a2d4-fb0475223943-metrics-client-ca\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.186867 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.186852 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-tls\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.187048 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.186886 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-accelerators-collector-config\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.187048 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:10:06.187028 2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 16 09:10:06.187155 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.187078 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-textfile\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.187155 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.187122 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b83a288-4615-4ee7-a2d4-fb0475223943-sys\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.187256 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.187171 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b83a288-4615-4ee7-a2d4-fb0475223943-root\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.187256 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.187208 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-wtmp\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.187370 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.187350 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-accelerators-collector-config\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.187442 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.187366 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-wtmp\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.187442 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.187410 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b83a288-4615-4ee7-a2d4-fb0475223943-sys\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.187442 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.187416 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b83a288-4615-4ee7-a2d4-fb0475223943-root\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.187590 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:10:06.187516 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-tls podName:8b83a288-4615-4ee7-a2d4-fb0475223943 nodeName:}" failed. No retries permitted until 2026-07-16 09:10:06.687496371 +0000 UTC m=+282.097044624 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-tls") pod "node-exporter-7996v" (UID: "8b83a288-4615-4ee7-a2d4-fb0475223943") : secret "node-exporter-tls" not found Jul 16 09:10:06.187651 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.187631 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86cdc97c94-vs4mc"] Jul 16 09:10:06.187703 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.187658 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-textfile\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.188197 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.188179 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.190049 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.190034 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.192883 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.192862 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 09:10:06.192986 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.192867 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 09:10:06.192986 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.192899 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 09:10:06.192986 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.192866 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 09:10:06.193318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.193166 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 09:10:06.193318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.193203 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 09:10:06.193575 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.193560 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mzcd6\"" Jul 16 09:10:06.193648 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.193605 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 09:10:06.198117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.198100 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 09:10:06.203048 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.203023 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgxx6\" (UniqueName: \"kubernetes.io/projected/8b83a288-4615-4ee7-a2d4-fb0475223943-kube-api-access-sgxx6\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.203790 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.203771 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86cdc97c94-vs4mc"] Jul 16 09:10:06.288472 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.288434 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff918822-58d7-49cf-a8a5-c860e147af23-console-serving-cert\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.288651 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.288506 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-console-config\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.288651 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.288556 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-service-ca\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.288651 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.288585 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff918822-58d7-49cf-a8a5-c860e147af23-console-oauth-config\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.288651 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.288619 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-oauth-serving-cert\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.288879 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.288690 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sptmq\" (UniqueName: \"kubernetes.io/projected/ff918822-58d7-49cf-a8a5-c860e147af23-kube-api-access-sptmq\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.288879 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.288728 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-trusted-ca-bundle\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.389877 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.389788 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-console-config\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.389877 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.389844 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-service-ca\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.389877 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.389862 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff918822-58d7-49cf-a8a5-c860e147af23-console-oauth-config\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.390130 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.389884 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-oauth-serving-cert\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.390130 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.389923 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sptmq\" (UniqueName: \"kubernetes.io/projected/ff918822-58d7-49cf-a8a5-c860e147af23-kube-api-access-sptmq\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.390130 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.389966 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-trusted-ca-bundle\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.390130 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.390013 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff918822-58d7-49cf-a8a5-c860e147af23-console-serving-cert\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.390766 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.390653 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-console-config\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.390766 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.390688 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-oauth-serving-cert\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.390766 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.390691 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-service-ca\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.391082 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.391063 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-trusted-ca-bundle\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.392905 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.392887 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff918822-58d7-49cf-a8a5-c860e147af23-console-oauth-config\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.393085 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.393066 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff918822-58d7-49cf-a8a5-c860e147af23-console-serving-cert\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.399528 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.399501 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sptmq\" (UniqueName: \"kubernetes.io/projected/ff918822-58d7-49cf-a8a5-c860e147af23-kube-api-access-sptmq\") pod \"console-86cdc97c94-vs4mc\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.515904 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.515875 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:06.656140 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.656120 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86cdc97c94-vs4mc"] Jul 16 09:10:06.658215 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:10:06.658187 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff918822_58d7_49cf_a8a5_c860e147af23.slice/crio-82fa716d84236c792b46763b5a8a9cf3158e57c9f4bf0e62f6b2a72af10cfe90 WatchSource:0}: Error finding container 82fa716d84236c792b46763b5a8a9cf3158e57c9f4bf0e62f6b2a72af10cfe90: Status 404 returned error can't find the container with id 82fa716d84236c792b46763b5a8a9cf3158e57c9f4bf0e62f6b2a72af10cfe90 Jul 16 09:10:06.692863 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.692838 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-tls\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.695059 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.695033 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b83a288-4615-4ee7-a2d4-fb0475223943-node-exporter-tls\") pod \"node-exporter-7996v\" (UID: \"8b83a288-4615-4ee7-a2d4-fb0475223943\") " pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.760633 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:10:06.760589 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-szwcd" podUID="f1ef4d9f-e004-434d-af44-a51c96bfeb32" Jul 16 09:10:06.968435 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:06.968336 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7996v" Jul 16 09:10:06.982783 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:10:06.982749 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b83a288_4615_4ee7_a2d4_fb0475223943.slice/crio-f3e624d2a82188a5af05b6fcca173ab4f624a0d6f25315f77c223cea6dfb3455 WatchSource:0}: Error finding container f3e624d2a82188a5af05b6fcca173ab4f624a0d6f25315f77c223cea6dfb3455: Status 404 returned error can't find the container with id f3e624d2a82188a5af05b6fcca173ab4f624a0d6f25315f77c223cea6dfb3455 Jul 16 09:10:07.138927 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.138887 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7996v" event={"ID":"8b83a288-4615-4ee7-a2d4-fb0475223943","Type":"ContainerStarted","Data":"f3e624d2a82188a5af05b6fcca173ab4f624a0d6f25315f77c223cea6dfb3455"} Jul 16 09:10:07.140171 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.140153 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-szwcd" Jul 16 09:10:07.140256 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.140147 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86cdc97c94-vs4mc" event={"ID":"ff918822-58d7-49cf-a8a5-c860e147af23","Type":"ContainerStarted","Data":"82fa716d84236c792b46763b5a8a9cf3158e57c9f4bf0e62f6b2a72af10cfe90"} Jul 16 09:10:07.147503 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.147473 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 09:10:07.150951 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.150932 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.153380 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.153356 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 16 09:10:07.153571 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.153556 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 16 09:10:07.153646 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.153577 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 16 09:10:07.154026 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.153830 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 16 09:10:07.154026 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.153841 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 16 09:10:07.154026 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.153915 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 16 09:10:07.154026 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.153931 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 16 09:10:07.154343 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.154138 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 16 09:10:07.154343 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.154240 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h7z9s\"" Jul 16 09:10:07.154343 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.154307 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 16 09:10:07.169644 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.169626 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 09:10:07.198841 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.198794 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.198977 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.198850 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d00dd833-e80c-4da1-92d1-c169c10309a4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.198977 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.198910 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.198977 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.198944 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d00dd833-e80c-4da1-92d1-c169c10309a4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.198977 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.198971 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.199196 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.198997 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d00dd833-e80c-4da1-92d1-c169c10309a4-config-out\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.199196 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.199025 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-config-volume\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.199196 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.199131 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f7jg\" (UniqueName: \"kubernetes.io/projected/d00dd833-e80c-4da1-92d1-c169c10309a4-kube-api-access-7f7jg\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.199196 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.199178 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-web-config\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.199385 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.199263 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d00dd833-e80c-4da1-92d1-c169c10309a4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.199385 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.199303 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d00dd833-e80c-4da1-92d1-c169c10309a4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.199385 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.199335 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.199646 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.199397 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.300518 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.300487 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.300712 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.300530 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d00dd833-e80c-4da1-92d1-c169c10309a4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.300712 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.300550 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.300712 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.300567 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d00dd833-e80c-4da1-92d1-c169c10309a4-config-out\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.300712 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.300591 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-config-volume\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.300712 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.300615 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f7jg\" (UniqueName: \"kubernetes.io/projected/d00dd833-e80c-4da1-92d1-c169c10309a4-kube-api-access-7f7jg\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.300712 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.300642 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-web-config\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.300712 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.300678 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d00dd833-e80c-4da1-92d1-c169c10309a4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.300712 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.300700 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d00dd833-e80c-4da1-92d1-c169c10309a4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.301122 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.300732 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.301122 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.300767 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.301122 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.300858 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.301122 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.300963 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d00dd833-e80c-4da1-92d1-c169c10309a4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.301122 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.300984 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d00dd833-e80c-4da1-92d1-c169c10309a4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.301387 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:10:07.300662 2567 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 16 09:10:07.302287 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.302238 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d00dd833-e80c-4da1-92d1-c169c10309a4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.302593 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.302522 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d00dd833-e80c-4da1-92d1-c169c10309a4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.302699 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:10:07.302687 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-main-tls podName:d00dd833-e80c-4da1-92d1-c169c10309a4 nodeName:}" failed. No retries permitted until 2026-07-16 09:10:07.80150118 +0000 UTC m=+283.211049423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "d00dd833-e80c-4da1-92d1-c169c10309a4") : secret "alertmanager-main-tls" not found Jul 16 09:10:07.303765 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.303740 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d00dd833-e80c-4da1-92d1-c169c10309a4-config-out\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.304231 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.304207 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.305265 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.305177 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.305265 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.305221 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-config-volume\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.305406 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.305268 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d00dd833-e80c-4da1-92d1-c169c10309a4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.305846 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.305811 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-web-config\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.306165 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.306147 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.306231 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.306211 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.310256 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.310233 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f7jg\" (UniqueName: \"kubernetes.io/projected/d00dd833-e80c-4da1-92d1-c169c10309a4-kube-api-access-7f7jg\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.806256 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.806197 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:07.809344 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:07.809302 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:08.062504 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:08.062436 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:10:08.147326 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:08.147289 2567 generic.go:356] "Generic (PLEG): container finished" podID="8b83a288-4615-4ee7-a2d4-fb0475223943" containerID="ba14f2faeae85ef3622be7d86e34000dd543dda40ee99a84195225a0c3042ec9" exitCode=0 Jul 16 09:10:08.147750 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:08.147394 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7996v" event={"ID":"8b83a288-4615-4ee7-a2d4-fb0475223943","Type":"ContainerDied","Data":"ba14f2faeae85ef3622be7d86e34000dd543dda40ee99a84195225a0c3042ec9"} Jul 16 09:10:08.226517 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:08.226484 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 09:10:08.229591 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:10:08.229566 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd00dd833_e80c_4da1_92d1_c169c10309a4.slice/crio-e71a3378189da3c435b27c1208537c1d7801d33f9ac0bebf59c5933e6bcddacd WatchSource:0}: Error finding container e71a3378189da3c435b27c1208537c1d7801d33f9ac0bebf59c5933e6bcddacd: Status 404 returned error can't find the container with id e71a3378189da3c435b27c1208537c1d7801d33f9ac0bebf59c5933e6bcddacd Jul 16 09:10:09.153217 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.153173 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerStarted","Data":"e71a3378189da3c435b27c1208537c1d7801d33f9ac0bebf59c5933e6bcddacd"} Jul 16 09:10:09.156130 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.156100 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7996v" event={"ID":"8b83a288-4615-4ee7-a2d4-fb0475223943","Type":"ContainerStarted","Data":"53a14eec8f5bbac471539268fca16aeea1242af8ec559c9e457641c476014c53"} Jul 16 09:10:09.156275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.156151 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7996v" event={"ID":"8b83a288-4615-4ee7-a2d4-fb0475223943","Type":"ContainerStarted","Data":"1c5e6f1ee0dcbd7a1ad55e0a0c8c9b1a14273b55ec348562f056594de793c723"} Jul 16 09:10:09.169991 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.169963 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-fc66995cd-gwscm"] Jul 16 09:10:09.173290 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.173271 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.176146 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.176018 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 16 09:10:09.176146 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.176019 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 16 09:10:09.176311 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.176254 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bnm4qoqnte20f\"" Jul 16 09:10:09.176373 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.176328 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 16 09:10:09.176621 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.176606 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 16 09:10:09.176795 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.176604 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 16 09:10:09.176795 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.176755 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-n4kcr\"" Jul 16 09:10:09.182426 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.182383 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7996v" podStartSLOduration=2.202699292 podStartE2EDuration="3.182365287s" totalImagesPullingTime="902.044368ms" totalInitContainerRuntime="77.621627ms" isStatefulPod=true podCreationTimestamp="2026-07-16 09:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:10:09.180331141 +0000 UTC m=+284.589879432" watchObservedRunningTime="2026-07-16 09:10:09.182365287 +0000 UTC m=+284.591913564" Jul 16 09:10:09.191583 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.191559 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-fc66995cd-gwscm"] Jul 16 09:10:09.219463 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.219431 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.219634 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.219549 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.219634 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.219588 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.219952 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.219913 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdpmc\" (UniqueName: \"kubernetes.io/projected/f3db39f0-a2c4-4cf3-830a-0fa307cba303-kube-api-access-vdpmc\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.220052 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.219974 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.220052 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.220012 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3db39f0-a2c4-4cf3-830a-0fa307cba303-metrics-client-ca\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.220052 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.220047 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-grpc-tls\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.220218 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.220154 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-tls\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.321411 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.321375 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.321596 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.321434 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.321596 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.321464 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.321596 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.321538 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdpmc\" (UniqueName: \"kubernetes.io/projected/f3db39f0-a2c4-4cf3-830a-0fa307cba303-kube-api-access-vdpmc\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.321596 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.321565 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.321596 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.321589 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3db39f0-a2c4-4cf3-830a-0fa307cba303-metrics-client-ca\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.321867 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.321612 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-grpc-tls\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.321867 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.321676 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-tls\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.323039 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.322972 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3db39f0-a2c4-4cf3-830a-0fa307cba303-metrics-client-ca\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.325011 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.324985 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.325964 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.325882 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.326434 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.326392 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.326640 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.326615 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-tls\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.327277 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.327251 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.328218 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.328194 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3db39f0-a2c4-4cf3-830a-0fa307cba303-secret-grpc-tls\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.335022 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.335003 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdpmc\" (UniqueName: \"kubernetes.io/projected/f3db39f0-a2c4-4cf3-830a-0fa307cba303-kube-api-access-vdpmc\") pod \"thanos-querier-fc66995cd-gwscm\" (UID: \"f3db39f0-a2c4-4cf3-830a-0fa307cba303\") " pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.485720 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.485636 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:09.826428 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.826389 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:10:09.829031 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.828689 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f1ef4d9f-e004-434d-af44-a51c96bfeb32-metrics-tls\") pod \"dns-default-szwcd\" (UID: \"f1ef4d9f-e004-434d-af44-a51c96bfeb32\") " pod="openshift-dns/dns-default-szwcd" Jul 16 09:10:09.843584 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.843559 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xmnzs\"" Jul 16 09:10:09.850669 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.850636 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-szwcd" Jul 16 09:10:09.991987 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:09.991961 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-fc66995cd-gwscm"] Jul 16 09:10:09.994840 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:10:09.994781 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3db39f0_a2c4_4cf3_830a_0fa307cba303.slice/crio-7003afc62845c9675e1651ac99e78145187a6613b4ad4b37be0f4629659d103f WatchSource:0}: Error finding container 7003afc62845c9675e1651ac99e78145187a6613b4ad4b37be0f4629659d103f: Status 404 returned error can't find the container with id 7003afc62845c9675e1651ac99e78145187a6613b4ad4b37be0f4629659d103f Jul 16 09:10:10.014754 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.014722 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-szwcd"] Jul 16 09:10:10.019044 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:10:10.019010 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ef4d9f_e004_434d_af44_a51c96bfeb32.slice/crio-352556e70fc4dd78cc6c57041a0f4b9b52d624a510d01f655c639705f4e4515c WatchSource:0}: Error finding container 352556e70fc4dd78cc6c57041a0f4b9b52d624a510d01f655c639705f4e4515c: Status 404 returned error can't find the container with id 352556e70fc4dd78cc6c57041a0f4b9b52d624a510d01f655c639705f4e4515c Jul 16 09:10:10.160962 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.160886 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-szwcd" event={"ID":"f1ef4d9f-e004-434d-af44-a51c96bfeb32","Type":"ContainerStarted","Data":"352556e70fc4dd78cc6c57041a0f4b9b52d624a510d01f655c639705f4e4515c"} Jul 16 09:10:10.162387 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.162356 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86cdc97c94-vs4mc" event={"ID":"ff918822-58d7-49cf-a8a5-c860e147af23","Type":"ContainerStarted","Data":"4eaca9c1cfe76869c7bae94cc1fc9e3ea5090a1495a4841ae18f263ea6c44450"} Jul 16 09:10:10.163483 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.163460 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" event={"ID":"f3db39f0-a2c4-4cf3-830a-0fa307cba303","Type":"ContainerStarted","Data":"7003afc62845c9675e1651ac99e78145187a6613b4ad4b37be0f4629659d103f"} Jul 16 09:10:10.185355 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.185313 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86cdc97c94-vs4mc" podStartSLOduration=0.931610473 podStartE2EDuration="4.185297915s" totalImagesPullingTime="3.253687442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:10:10.183863924 +0000 UTC m=+285.593412212" watchObservedRunningTime="2026-07-16 09:10:10.185297915 +0000 UTC m=+285.594846173" Jul 16 09:10:10.568627 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.567984 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6dfbf977df-nq9ln"] Jul 16 09:10:10.573036 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.573005 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.576328 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.576304 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 16 09:10:10.576429 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.576366 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 16 09:10:10.576429 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.576403 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 16 09:10:10.576547 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.576436 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 16 09:10:10.576547 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.576448 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-9lbia0603kpal\"" Jul 16 09:10:10.576651 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.576605 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qn6pz\"" Jul 16 09:10:10.581944 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.581909 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6dfbf977df-nq9ln"] Jul 16 09:10:10.634090 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.634068 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a4816df6-0388-4b69-95d1-8f5e6fd3566b-metrics-server-audit-profiles\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.634199 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.634102 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a4816df6-0388-4b69-95d1-8f5e6fd3566b-audit-log\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.634199 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.634124 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkpqw\" (UniqueName: \"kubernetes.io/projected/a4816df6-0388-4b69-95d1-8f5e6fd3566b-kube-api-access-fkpqw\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.634199 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.634152 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4816df6-0388-4b69-95d1-8f5e6fd3566b-client-ca-bundle\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.634331 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.634215 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a4816df6-0388-4b69-95d1-8f5e6fd3566b-secret-metrics-server-tls\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.634331 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.634251 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a4816df6-0388-4b69-95d1-8f5e6fd3566b-secret-metrics-server-client-certs\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.634331 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.634285 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4816df6-0388-4b69-95d1-8f5e6fd3566b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.734741 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.734701 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a4816df6-0388-4b69-95d1-8f5e6fd3566b-audit-log\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.734741 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.734740 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkpqw\" (UniqueName: \"kubernetes.io/projected/a4816df6-0388-4b69-95d1-8f5e6fd3566b-kube-api-access-fkpqw\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.734993 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.734778 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4816df6-0388-4b69-95d1-8f5e6fd3566b-client-ca-bundle\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.734993 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.734839 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a4816df6-0388-4b69-95d1-8f5e6fd3566b-secret-metrics-server-tls\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.734993 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.734881 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a4816df6-0388-4b69-95d1-8f5e6fd3566b-secret-metrics-server-client-certs\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.734993 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.734935 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4816df6-0388-4b69-95d1-8f5e6fd3566b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.734993 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.734985 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a4816df6-0388-4b69-95d1-8f5e6fd3566b-metrics-server-audit-profiles\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.735252 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.735163 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a4816df6-0388-4b69-95d1-8f5e6fd3566b-audit-log\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.735668 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.735628 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4816df6-0388-4b69-95d1-8f5e6fd3566b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.735812 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.735795 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a4816df6-0388-4b69-95d1-8f5e6fd3566b-metrics-server-audit-profiles\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.737381 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.737346 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4816df6-0388-4b69-95d1-8f5e6fd3566b-client-ca-bundle\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.737628 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.737607 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a4816df6-0388-4b69-95d1-8f5e6fd3566b-secret-metrics-server-tls\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.737752 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.737733 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a4816df6-0388-4b69-95d1-8f5e6fd3566b-secret-metrics-server-client-certs\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.745841 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.745805 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkpqw\" (UniqueName: \"kubernetes.io/projected/a4816df6-0388-4b69-95d1-8f5e6fd3566b-kube-api-access-fkpqw\") pod \"metrics-server-6dfbf977df-nq9ln\" (UID: \"a4816df6-0388-4b69-95d1-8f5e6fd3566b\") " pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.818224 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.818145 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n"] Jul 16 09:10:10.822830 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.822798 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n" Jul 16 09:10:10.825855 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.825832 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 16 09:10:10.826083 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.826069 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hdznv\"" Jul 16 09:10:10.832423 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.832402 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n"] Jul 16 09:10:10.911350 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.911315 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:10.935923 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.935892 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-2dx8n\" (UID: \"8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n" Jul 16 09:10:10.936062 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:10.935950 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qds7h\" (UniqueName: \"kubernetes.io/projected/8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f-kube-api-access-qds7h\") pod \"monitoring-plugin-5f68df7f58-2dx8n\" (UID: \"8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n" Jul 16 09:10:11.037377 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:11.037343 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-2dx8n\" (UID: \"8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n" Jul 16 09:10:11.037542 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:11.037394 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qds7h\" (UniqueName: \"kubernetes.io/projected/8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f-kube-api-access-qds7h\") pod \"monitoring-plugin-5f68df7f58-2dx8n\" (UID: \"8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n" Jul 16 09:10:11.039834 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:11.039795 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-2dx8n\" (UID: \"8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n" Jul 16 09:10:11.050007 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:11.049986 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qds7h\" (UniqueName: \"kubernetes.io/projected/8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f-kube-api-access-qds7h\") pod \"monitoring-plugin-5f68df7f58-2dx8n\" (UID: \"8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n" Jul 16 09:10:11.058730 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:11.058709 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6dfbf977df-nq9ln"] Jul 16 09:10:11.060932 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:10:11.060905 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4816df6_0388_4b69_95d1_8f5e6fd3566b.slice/crio-ded61ecb3d9fe50238ed73b459125a46721909ff3ee177ea5712e62a49d27a71 WatchSource:0}: Error finding container ded61ecb3d9fe50238ed73b459125a46721909ff3ee177ea5712e62a49d27a71: Status 404 returned error can't find the container with id ded61ecb3d9fe50238ed73b459125a46721909ff3ee177ea5712e62a49d27a71 Jul 16 09:10:11.132562 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:11.132482 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n" Jul 16 09:10:11.168652 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:11.168618 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" event={"ID":"a4816df6-0388-4b69-95d1-8f5e6fd3566b","Type":"ContainerStarted","Data":"ded61ecb3d9fe50238ed73b459125a46721909ff3ee177ea5712e62a49d27a71"} Jul 16 09:10:11.170114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:11.170089 2567 generic.go:356] "Generic (PLEG): container finished" podID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerID="4835322c040cf7813dc22846944818a7af2e6c5b42cee2b872391d60c200ed57" exitCode=0 Jul 16 09:10:11.170229 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:11.170179 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerDied","Data":"4835322c040cf7813dc22846944818a7af2e6c5b42cee2b872391d60c200ed57"} Jul 16 09:10:11.285896 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:11.285811 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n"] Jul 16 09:10:11.288426 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:10:11.288403 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf7f25f_65cb_4bf3_b422_b1a0ebbc339f.slice/crio-da81475f6928bf99ffae9d6c7d692d868909f040244cacd5b2125647e9bcf031 WatchSource:0}: Error finding container da81475f6928bf99ffae9d6c7d692d868909f040244cacd5b2125647e9bcf031: Status 404 returned error can't find the container with id da81475f6928bf99ffae9d6c7d692d868909f040244cacd5b2125647e9bcf031 Jul 16 09:10:12.177931 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.177854 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n" event={"ID":"8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f","Type":"ContainerStarted","Data":"da81475f6928bf99ffae9d6c7d692d868909f040244cacd5b2125647e9bcf031"} Jul 16 09:10:12.329606 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.329531 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 09:10:12.336469 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.334890 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.338691 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.338656 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 16 09:10:12.338809 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.338775 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 16 09:10:12.338890 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.338881 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 16 09:10:12.339108 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.338965 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 16 09:10:12.339108 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.339097 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 16 09:10:12.340308 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.340078 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 16 09:10:12.340308 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.340123 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-53q61v5abv5u\"" Jul 16 09:10:12.340464 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.340380 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 16 09:10:12.340550 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.340510 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 16 09:10:12.340613 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.340562 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 16 09:10:12.340726 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.340707 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 16 09:10:12.340781 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.340738 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 16 09:10:12.341211 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.340944 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mlpvb\"" Jul 16 09:10:12.341211 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.340950 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 16 09:10:12.341758 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.341398 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 16 09:10:12.345304 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.344878 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 16 09:10:12.351375 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.351350 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.351592 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.351576 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.351896 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.351839 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352101 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352083 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352260 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352174 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-config\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352260 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352210 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w5ns\" (UniqueName: \"kubernetes.io/projected/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-kube-api-access-9w5ns\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352260 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352237 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352431 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352260 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352431 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352314 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352431 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352339 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352431 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352374 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352431 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352399 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352673 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352437 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352673 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352471 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352673 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352525 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-config-out\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352673 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352554 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352673 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352578 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352673 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352647 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352980 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352690 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.352980 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.352722 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-web-config\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.354519 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.354501 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 09:10:12.454953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.453706 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-config\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.454953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.453754 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w5ns\" (UniqueName: \"kubernetes.io/projected/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-kube-api-access-9w5ns\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.454953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.453800 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.454953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.453847 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.454953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.453893 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.454953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.453920 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.454953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.453944 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.454953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.453971 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.454953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.454007 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.454953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.454041 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.454953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.454099 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-config-out\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.454953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.454127 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.454953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.454151 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.454953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.454211 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.455702 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.454242 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.455702 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.454270 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-web-config\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.455702 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.454295 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.455702 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.454328 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.455702 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.454378 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.455702 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.454425 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.455702 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.455343 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.458443 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.456556 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.458443 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.457525 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.459983 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.459688 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.461730 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.460718 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.461730 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.461212 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.461730 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:10:12.461365 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-trusted-ca-bundle podName:a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c nodeName:}" failed. No retries permitted until 2026-07-16 09:10:12.961346283 +0000 UTC m=+288.370894537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-trusted-ca-bundle") pod "prometheus-k8s-0" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c") : configmap references non-existent config key: ca-bundle.crt Jul 16 09:10:12.461730 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.461418 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.466041 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.465989 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-config-out\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.467211 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.467159 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.467473 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.467377 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.467473 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.467463 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.468193 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.468045 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.468193 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.468145 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.468193 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.468160 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.476275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.476250 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-web-config\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.483888 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.477245 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-config\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.483888 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.478513 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.483888 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.480411 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w5ns\" (UniqueName: \"kubernetes.io/projected/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-kube-api-access-9w5ns\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:12.483888 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:12.481557 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:13.061957 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:13.061777 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:13.062800 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:13.062770 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:13.251487 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:13.251446 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:14.851850 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:14.851621 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 09:10:14.869338 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:10:14.869309 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b356f7_80d7_4a67_bf0d_fa79cd3a3b8c.slice/crio-2b72ea653a145fd43e7300922f45db4693aa30a96e42e889df5057b3f7007a37 WatchSource:0}: Error finding container 2b72ea653a145fd43e7300922f45db4693aa30a96e42e889df5057b3f7007a37: Status 404 returned error can't find the container with id 2b72ea653a145fd43e7300922f45db4693aa30a96e42e889df5057b3f7007a37 Jul 16 09:10:15.011943 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.011604 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86cdc97c94-vs4mc"] Jul 16 09:10:15.194104 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.194070 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" event={"ID":"f3db39f0-a2c4-4cf3-830a-0fa307cba303","Type":"ContainerStarted","Data":"c9610304a13a46a076b6e2739a1d82a9177c170efaf34872aaca232b68103b72"} Jul 16 09:10:15.194247 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.194144 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" event={"ID":"f3db39f0-a2c4-4cf3-830a-0fa307cba303","Type":"ContainerStarted","Data":"1781170a74d04c4984ccc49247f7781a609fb78195ef2104b25e22c8c327c919"} Jul 16 09:10:15.194247 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.194184 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" event={"ID":"f3db39f0-a2c4-4cf3-830a-0fa307cba303","Type":"ContainerStarted","Data":"e891f84e26eeceb82f0d844714bbf8bdad248ddc8f797a30c0a8785688908ca1"} Jul 16 09:10:15.198333 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.198294 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerStarted","Data":"51de7aaa5f4bd4a71f024966413b973aa38d50ef1ebfa95c288c7d2ff8aa9112"} Jul 16 09:10:15.198464 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.198372 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerStarted","Data":"e541dd66102231b2e7de21eef1c19480f46a2e3d3af37aa2868b46687e29e78f"} Jul 16 09:10:15.198464 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.198419 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerStarted","Data":"d073ba0a71a7ea8dc40a76382e4f09217a8034ea4229208a0cdee3bb2a50f455"} Jul 16 09:10:15.198464 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.198457 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerStarted","Data":"ade868a9b20cb4ce6db89c0deaf47bef0af78ccb54d19c15f13d8d84d60206ad"} Jul 16 09:10:15.198611 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.198494 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerStarted","Data":"717dc7583bc215b1ac8413a8fbe2f77a36b6a9ab80dedaa1c972b20bc56e4670"} Jul 16 09:10:15.200525 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.200502 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-szwcd" event={"ID":"f1ef4d9f-e004-434d-af44-a51c96bfeb32","Type":"ContainerStarted","Data":"bfcd60b31e8a5da4bc198d31ec7eaa83c44ea7cc7533836eb19aad99448340a9"} Jul 16 09:10:15.200625 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.200548 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-szwcd" event={"ID":"f1ef4d9f-e004-434d-af44-a51c96bfeb32","Type":"ContainerStarted","Data":"5f04a919a64db5f38873c5661fde06ffae1da1bca228469c63742e7c9c206483"} Jul 16 09:10:15.200702 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.200633 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-szwcd" Jul 16 09:10:15.202099 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.202076 2567 generic.go:356] "Generic (PLEG): container finished" podID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerID="652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d" exitCode=0 Jul 16 09:10:15.202197 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.202171 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerDied","Data":"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d"} Jul 16 09:10:15.202254 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.202237 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerStarted","Data":"2b72ea653a145fd43e7300922f45db4693aa30a96e42e889df5057b3f7007a37"} Jul 16 09:10:15.204056 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.204034 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" event={"ID":"a4816df6-0388-4b69-95d1-8f5e6fd3566b","Type":"ContainerStarted","Data":"39b9ee82a81b03946d8d2e293987f91e5543351d3338127fd7ef4591d87e7c67"} Jul 16 09:10:15.208918 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.208895 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n" event={"ID":"8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f","Type":"ContainerStarted","Data":"60ae73f0abafe10b03805f10472913946fce19ac10f13d84ff749c44ce4d44a2"} Jul 16 09:10:15.209090 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.209069 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n" Jul 16 09:10:15.215427 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.215411 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n" Jul 16 09:10:15.219509 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.219473 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-szwcd" podStartSLOduration=251.55641125 podStartE2EDuration="4m16.219460134s" totalImagesPullingTime="4.663048884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:05:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:10:15.217629235 +0000 UTC m=+290.627177495" watchObservedRunningTime="2026-07-16 09:10:15.219460134 +0000 UTC m=+290.629008370" Jul 16 09:10:15.237932 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.237887 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" podStartSLOduration=1.6240458100000001 podStartE2EDuration="5.237874931s" totalImagesPullingTime="3.613829121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:10:15.236194998 +0000 UTC m=+290.645743267" watchObservedRunningTime="2026-07-16 09:10:15.237874931 +0000 UTC m=+290.647423168" Jul 16 09:10:15.282255 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:15.282206 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-2dx8n" podStartSLOduration=1.895889869 podStartE2EDuration="5.282192772s" totalImagesPullingTime="3.386302903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:10:15.281518164 +0000 UTC m=+290.691066424" watchObservedRunningTime="2026-07-16 09:10:15.282192772 +0000 UTC m=+290.691741030" Jul 16 09:10:16.516028 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:16.515989 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:16.931089 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:16.930959 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" podUID="d4cf179b-d79d-44c6-a486-6cc872cc84ff" containerName="registry" containerID="cri-o://ed2dbd4f2f9f4f2325f6daaf94cf8088ecc09e667688372bc504cde1abc31420" gracePeriod=30 Jul 16 09:10:17.220267 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:17.220223 2567 generic.go:356] "Generic (PLEG): container finished" podID="d4cf179b-d79d-44c6-a486-6cc872cc84ff" containerID="ed2dbd4f2f9f4f2325f6daaf94cf8088ecc09e667688372bc504cde1abc31420" exitCode=0 Jul 16 09:10:17.220431 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:17.220289 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" event={"ID":"d4cf179b-d79d-44c6-a486-6cc872cc84ff","Type":"ContainerDied","Data":"ed2dbd4f2f9f4f2325f6daaf94cf8088ecc09e667688372bc504cde1abc31420"} Jul 16 09:10:18.291841 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.291805 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:10:18.416771 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.416744 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4cf179b-d79d-44c6-a486-6cc872cc84ff-image-registry-private-configuration\") pod \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " Jul 16 09:10:18.416967 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.416803 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptck5\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-kube-api-access-ptck5\") pod \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " Jul 16 09:10:18.416967 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.416874 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-certificates\") pod \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " Jul 16 09:10:18.416967 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.416910 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls\") pod \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " Jul 16 09:10:18.417139 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.416967 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4cf179b-d79d-44c6-a486-6cc872cc84ff-trusted-ca\") pod \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " Jul 16 09:10:18.417139 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.417028 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-bound-sa-token\") pod \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " Jul 16 09:10:18.417139 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.417071 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4cf179b-d79d-44c6-a486-6cc872cc84ff-ca-trust-extracted\") pod \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " Jul 16 09:10:18.417139 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.417128 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4cf179b-d79d-44c6-a486-6cc872cc84ff-installation-pull-secrets\") pod \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\" (UID: \"d4cf179b-d79d-44c6-a486-6cc872cc84ff\") " Jul 16 09:10:18.417797 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.417695 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d4cf179b-d79d-44c6-a486-6cc872cc84ff" (UID: "d4cf179b-d79d-44c6-a486-6cc872cc84ff"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:10:18.420984 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.418377 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4cf179b-d79d-44c6-a486-6cc872cc84ff-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d4cf179b-d79d-44c6-a486-6cc872cc84ff" (UID: "d4cf179b-d79d-44c6-a486-6cc872cc84ff"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:10:18.420984 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.420577 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4cf179b-d79d-44c6-a486-6cc872cc84ff-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d4cf179b-d79d-44c6-a486-6cc872cc84ff" (UID: "d4cf179b-d79d-44c6-a486-6cc872cc84ff"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:10:18.421498 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.421171 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-kube-api-access-ptck5" (OuterVolumeSpecName: "kube-api-access-ptck5") pod "d4cf179b-d79d-44c6-a486-6cc872cc84ff" (UID: "d4cf179b-d79d-44c6-a486-6cc872cc84ff"). InnerVolumeSpecName "kube-api-access-ptck5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 09:10:18.421498 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.421337 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d4cf179b-d79d-44c6-a486-6cc872cc84ff" (UID: "d4cf179b-d79d-44c6-a486-6cc872cc84ff"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 09:10:18.421839 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.421749 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d4cf179b-d79d-44c6-a486-6cc872cc84ff" (UID: "d4cf179b-d79d-44c6-a486-6cc872cc84ff"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 09:10:18.421989 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.421967 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4cf179b-d79d-44c6-a486-6cc872cc84ff-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d4cf179b-d79d-44c6-a486-6cc872cc84ff" (UID: "d4cf179b-d79d-44c6-a486-6cc872cc84ff"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:10:18.428889 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.428863 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4cf179b-d79d-44c6-a486-6cc872cc84ff-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d4cf179b-d79d-44c6-a486-6cc872cc84ff" (UID: "d4cf179b-d79d-44c6-a486-6cc872cc84ff"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 09:10:18.519255 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.519154 2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4cf179b-d79d-44c6-a486-6cc872cc84ff-ca-trust-extracted\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:18.519255 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.519185 2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4cf179b-d79d-44c6-a486-6cc872cc84ff-installation-pull-secrets\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:18.519255 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.519203 2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4cf179b-d79d-44c6-a486-6cc872cc84ff-image-registry-private-configuration\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:18.519255 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.519216 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptck5\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-kube-api-access-ptck5\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:18.519255 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.519229 2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-certificates\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:18.519255 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.519242 2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-registry-tls\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:18.519255 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.519255 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4cf179b-d79d-44c6-a486-6cc872cc84ff-trusted-ca\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:18.519543 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:18.519267 2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4cf179b-d79d-44c6-a486-6cc872cc84ff-bound-sa-token\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:19.232363 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.232310 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerStarted","Data":"5c09336cd59eca33792e3abc2b4872c5460c27518c365a8ba03fc622d3354806"} Jul 16 09:10:19.235433 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.235408 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerStarted","Data":"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e"} Jul 16 09:10:19.235571 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.235462 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerStarted","Data":"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d"} Jul 16 09:10:19.235571 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.235486 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerStarted","Data":"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c"} Jul 16 09:10:19.235571 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.235530 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerStarted","Data":"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6"} Jul 16 09:10:19.235571 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.235560 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerStarted","Data":"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50"} Jul 16 09:10:19.235847 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.235588 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerStarted","Data":"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30"} Jul 16 09:10:19.236591 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.236576 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" Jul 16 09:10:19.236682 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.236596 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5b8d48f6-r62kk" event={"ID":"d4cf179b-d79d-44c6-a486-6cc872cc84ff","Type":"ContainerDied","Data":"381bf294741a4a1e20393b643915168d94e8848dcace99fad511466cef6b2ee1"} Jul 16 09:10:19.236682 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.236632 2567 scope.go:117] "RemoveContainer" containerID="ed2dbd4f2f9f4f2325f6daaf94cf8088ecc09e667688372bc504cde1abc31420" Jul 16 09:10:19.239894 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.239872 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" event={"ID":"f3db39f0-a2c4-4cf3-830a-0fa307cba303","Type":"ContainerStarted","Data":"86a1692f4cc09bfa6a5611b09fdfbc18e9d68778ebe4e91964862da8a2243738"} Jul 16 09:10:19.239984 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.239928 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" event={"ID":"f3db39f0-a2c4-4cf3-830a-0fa307cba303","Type":"ContainerStarted","Data":"9777d27a20930a934a74fb9bb11b167b808eb29c0f4a5daa29b17d3b1192eda6"} Jul 16 09:10:19.239984 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.239956 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" event={"ID":"f3db39f0-a2c4-4cf3-830a-0fa307cba303","Type":"ContainerStarted","Data":"f820b7bb7cee48a9a0ae2677355fff421771a26e1eb0342ee6a3a993ff753de8"} Jul 16 09:10:19.240153 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.240134 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:19.246417 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.246402 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" Jul 16 09:10:19.264970 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.264922 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.266838791 podStartE2EDuration="12.264909145s" totalImagesPullingTime="8.9828601s" totalInitContainerRuntime="15.210254ms" isStatefulPod=false podCreationTimestamp="2026-07-16 09:10:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:10:19.262842031 +0000 UTC m=+294.672390330" watchObservedRunningTime="2026-07-16 09:10:19.264909145 +0000 UTC m=+294.674457404" Jul 16 09:10:19.281028 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.281002 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b5b8d48f6-r62kk"] Jul 16 09:10:19.286830 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.286795 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5b5b8d48f6-r62kk"] Jul 16 09:10:19.313587 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.313524 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.164090377 podStartE2EDuration="7.313511419s" totalImagesPullingTime="3.122953608s" totalInitContainerRuntime="26.467434ms" isStatefulPod=false podCreationTimestamp="2026-07-16 09:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:10:19.31194156 +0000 UTC m=+294.721489866" watchObservedRunningTime="2026-07-16 09:10:19.313511419 +0000 UTC m=+294.723059677" Jul 16 09:10:19.320096 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:10:19.320069 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ph7bq" podUID="9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd" Jul 16 09:10:19.324669 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.324646 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4cf179b-d79d-44c6-a486-6cc872cc84ff" path="/var/lib/kubelet/pods/d4cf179b-d79d-44c6-a486-6cc872cc84ff/volumes" Jul 16 09:10:19.334929 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:19.334529 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-fc66995cd-gwscm" podStartSLOduration=2.325260248 podStartE2EDuration="10.334511995s" totalImagesPullingTime="8.009251747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:10:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:10:19.331957773 +0000 UTC m=+294.741506066" watchObservedRunningTime="2026-07-16 09:10:19.334511995 +0000 UTC m=+294.744060259" Jul 16 09:10:23.252168 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:23.252135 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:10:25.080088 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:25.080058 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:10:25.080494 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:25.080299 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:10:25.090860 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:25.090840 2567 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 09:10:25.216955 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:25.216924 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-szwcd" Jul 16 09:10:30.912411 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:30.912310 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:30.912411 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:30.912408 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:31.292867 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:31.292804 2567 generic.go:356] "Generic (PLEG): container finished" podID="9fa2a7c4-5499-408f-9c98-acc1debc7be0" containerID="a2fff93f48211d82fd621d6da002a60d5cd4f86aed708f3bf2d5b3fdfda2bd81" exitCode=0 Jul 16 09:10:31.293028 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:31.292884 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" event={"ID":"9fa2a7c4-5499-408f-9c98-acc1debc7be0","Type":"ContainerDied","Data":"a2fff93f48211d82fd621d6da002a60d5cd4f86aed708f3bf2d5b3fdfda2bd81"} Jul 16 09:10:31.293228 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:31.293215 2567 scope.go:117] "RemoveContainer" containerID="a2fff93f48211d82fd621d6da002a60d5cd4f86aed708f3bf2d5b3fdfda2bd81" Jul 16 09:10:31.323118 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:31.323097 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:10:32.052216 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:32.052190 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kzpnm_d2d206a0-1603-4fd8-936a-dacba9979e0c/serve-healthcheck-canary/0.log" Jul 16 09:10:32.300883 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:32.300844 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-5gbht" event={"ID":"9fa2a7c4-5499-408f-9c98-acc1debc7be0","Type":"ContainerStarted","Data":"0de654074ce3d82b7315920b6ceb107d97536b14f7da738d94700ac167a5ab40"} Jul 16 09:10:40.032571 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.032501 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-86cdc97c94-vs4mc" podUID="ff918822-58d7-49cf-a8a5-c860e147af23" containerName="console" containerID="cri-o://4eaca9c1cfe76869c7bae94cc1fc9e3ea5090a1495a4841ae18f263ea6c44450" gracePeriod=15 Jul 16 09:10:40.269354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.269330 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:40.333127 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.333042 2567 generic.go:356] "Generic (PLEG): container finished" podID="ff918822-58d7-49cf-a8a5-c860e147af23" containerID="4eaca9c1cfe76869c7bae94cc1fc9e3ea5090a1495a4841ae18f263ea6c44450" exitCode=0 Jul 16 09:10:40.333127 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.333087 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86cdc97c94-vs4mc" Jul 16 09:10:40.333296 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.333127 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86cdc97c94-vs4mc" event={"ID":"ff918822-58d7-49cf-a8a5-c860e147af23","Type":"ContainerDied","Data":"4eaca9c1cfe76869c7bae94cc1fc9e3ea5090a1495a4841ae18f263ea6c44450"} Jul 16 09:10:40.333296 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.333177 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86cdc97c94-vs4mc" event={"ID":"ff918822-58d7-49cf-a8a5-c860e147af23","Type":"ContainerDied","Data":"82fa716d84236c792b46763b5a8a9cf3158e57c9f4bf0e62f6b2a72af10cfe90"} Jul 16 09:10:40.333296 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.333198 2567 scope.go:117] "RemoveContainer" containerID="4eaca9c1cfe76869c7bae94cc1fc9e3ea5090a1495a4841ae18f263ea6c44450" Jul 16 09:10:40.341685 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.341666 2567 scope.go:117] "RemoveContainer" containerID="4eaca9c1cfe76869c7bae94cc1fc9e3ea5090a1495a4841ae18f263ea6c44450" Jul 16 09:10:40.341968 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:10:40.341948 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4eaca9c1cfe76869c7bae94cc1fc9e3ea5090a1495a4841ae18f263ea6c44450\": container with ID starting with 4eaca9c1cfe76869c7bae94cc1fc9e3ea5090a1495a4841ae18f263ea6c44450 not found: ID does not exist" containerID="4eaca9c1cfe76869c7bae94cc1fc9e3ea5090a1495a4841ae18f263ea6c44450" Jul 16 09:10:40.342025 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.341977 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4eaca9c1cfe76869c7bae94cc1fc9e3ea5090a1495a4841ae18f263ea6c44450"} err="failed to get container status \"4eaca9c1cfe76869c7bae94cc1fc9e3ea5090a1495a4841ae18f263ea6c44450\": rpc error: code = NotFound desc = could not find container \"4eaca9c1cfe76869c7bae94cc1fc9e3ea5090a1495a4841ae18f263ea6c44450\": container with ID starting with 4eaca9c1cfe76869c7bae94cc1fc9e3ea5090a1495a4841ae18f263ea6c44450 not found: ID does not exist" Jul 16 09:10:40.425337 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.425301 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-service-ca\") pod \"ff918822-58d7-49cf-a8a5-c860e147af23\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " Jul 16 09:10:40.425509 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.425348 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-console-config\") pod \"ff918822-58d7-49cf-a8a5-c860e147af23\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " Jul 16 09:10:40.425509 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.425389 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff918822-58d7-49cf-a8a5-c860e147af23-console-serving-cert\") pod \"ff918822-58d7-49cf-a8a5-c860e147af23\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " Jul 16 09:10:40.425509 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.425435 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-oauth-serving-cert\") pod \"ff918822-58d7-49cf-a8a5-c860e147af23\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " Jul 16 09:10:40.425509 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.425460 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sptmq\" (UniqueName: \"kubernetes.io/projected/ff918822-58d7-49cf-a8a5-c860e147af23-kube-api-access-sptmq\") pod \"ff918822-58d7-49cf-a8a5-c860e147af23\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " Jul 16 09:10:40.425702 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.425523 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff918822-58d7-49cf-a8a5-c860e147af23-console-oauth-config\") pod \"ff918822-58d7-49cf-a8a5-c860e147af23\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " Jul 16 09:10:40.425702 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.425550 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-trusted-ca-bundle\") pod \"ff918822-58d7-49cf-a8a5-c860e147af23\" (UID: \"ff918822-58d7-49cf-a8a5-c860e147af23\") " Jul 16 09:10:40.426090 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.426046 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-console-config" (OuterVolumeSpecName: "console-config") pod "ff918822-58d7-49cf-a8a5-c860e147af23" (UID: "ff918822-58d7-49cf-a8a5-c860e147af23"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:10:40.426090 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.426070 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-service-ca" (OuterVolumeSpecName: "service-ca") pod "ff918822-58d7-49cf-a8a5-c860e147af23" (UID: "ff918822-58d7-49cf-a8a5-c860e147af23"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:10:40.426253 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.426128 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ff918822-58d7-49cf-a8a5-c860e147af23" (UID: "ff918822-58d7-49cf-a8a5-c860e147af23"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:10:40.426253 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.426223 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ff918822-58d7-49cf-a8a5-c860e147af23" (UID: "ff918822-58d7-49cf-a8a5-c860e147af23"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:10:40.427844 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.427788 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff918822-58d7-49cf-a8a5-c860e147af23-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ff918822-58d7-49cf-a8a5-c860e147af23" (UID: "ff918822-58d7-49cf-a8a5-c860e147af23"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:10:40.428103 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.428072 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff918822-58d7-49cf-a8a5-c860e147af23-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ff918822-58d7-49cf-a8a5-c860e147af23" (UID: "ff918822-58d7-49cf-a8a5-c860e147af23"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:10:40.428103 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.428085 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff918822-58d7-49cf-a8a5-c860e147af23-kube-api-access-sptmq" (OuterVolumeSpecName: "kube-api-access-sptmq") pod "ff918822-58d7-49cf-a8a5-c860e147af23" (UID: "ff918822-58d7-49cf-a8a5-c860e147af23"). InnerVolumeSpecName "kube-api-access-sptmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 09:10:40.526318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.526285 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-service-ca\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:40.526318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.526311 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-console-config\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:40.526318 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.526320 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff918822-58d7-49cf-a8a5-c860e147af23-console-serving-cert\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:40.526537 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.526329 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-oauth-serving-cert\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:40.526537 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.526338 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sptmq\" (UniqueName: \"kubernetes.io/projected/ff918822-58d7-49cf-a8a5-c860e147af23-kube-api-access-sptmq\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:40.526537 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.526347 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff918822-58d7-49cf-a8a5-c860e147af23-console-oauth-config\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:40.526537 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.526355 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff918822-58d7-49cf-a8a5-c860e147af23-trusted-ca-bundle\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:10:40.654492 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.654454 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86cdc97c94-vs4mc"] Jul 16 09:10:40.658674 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:40.658649 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86cdc97c94-vs4mc"] Jul 16 09:10:41.323786 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:41.323748 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff918822-58d7-49cf-a8a5-c860e147af23" path="/var/lib/kubelet/pods/ff918822-58d7-49cf-a8a5-c860e147af23/volumes" Jul 16 09:10:45.363896 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:45.363862 2567 generic.go:356] "Generic (PLEG): container finished" podID="dcfe3047-1320-4d55-adb2-636de23660c4" containerID="bd0afbabbd2cb5f734280f3d957d8926671afb785f61c81da84af7e5254c2898" exitCode=0 Jul 16 09:10:45.364270 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:45.363916 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" event={"ID":"dcfe3047-1320-4d55-adb2-636de23660c4","Type":"ContainerDied","Data":"bd0afbabbd2cb5f734280f3d957d8926671afb785f61c81da84af7e5254c2898"} Jul 16 09:10:45.364270 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:45.364249 2567 scope.go:117] "RemoveContainer" containerID="bd0afbabbd2cb5f734280f3d957d8926671afb785f61c81da84af7e5254c2898" Jul 16 09:10:46.378616 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:46.378578 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-kz47m" event={"ID":"dcfe3047-1320-4d55-adb2-636de23660c4","Type":"ContainerStarted","Data":"19c4c127c03c0c8a8c9fb2c0debff12b07456e13b80ca559e7b9d3e81cb3b52e"} Jul 16 09:10:50.917096 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:50.917066 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:10:50.920874 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:10:50.920849 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6dfbf977df-nq9ln" Jul 16 09:11:13.251921 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:13.251873 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:13.271888 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:13.271863 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:13.503747 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:13.503675 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:26.498685 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:26.498643 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 09:11:26.499345 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:26.499007 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="alertmanager" containerID="cri-o://717dc7583bc215b1ac8413a8fbe2f77a36b6a9ab80dedaa1c972b20bc56e4670" gracePeriod=120 Jul 16 09:11:26.499345 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:26.499068 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="kube-rbac-proxy-metric" containerID="cri-o://51de7aaa5f4bd4a71f024966413b973aa38d50ef1ebfa95c288c7d2ff8aa9112" gracePeriod=120 Jul 16 09:11:26.499345 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:26.499086 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="kube-rbac-proxy" containerID="cri-o://e541dd66102231b2e7de21eef1c19480f46a2e3d3af37aa2868b46687e29e78f" gracePeriod=120 Jul 16 09:11:26.499345 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:26.499112 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="config-reloader" containerID="cri-o://ade868a9b20cb4ce6db89c0deaf47bef0af78ccb54d19c15f13d8d84d60206ad" gracePeriod=120 Jul 16 09:11:26.499345 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:26.499129 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="prom-label-proxy" containerID="cri-o://5c09336cd59eca33792e3abc2b4872c5460c27518c365a8ba03fc622d3354806" gracePeriod=120 Jul 16 09:11:26.499345 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:26.499194 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="kube-rbac-proxy-web" containerID="cri-o://d073ba0a71a7ea8dc40a76382e4f09217a8034ea4229208a0cdee3bb2a50f455" gracePeriod=120 Jul 16 09:11:27.545338 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.545306 2567 generic.go:356] "Generic (PLEG): container finished" podID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerID="5c09336cd59eca33792e3abc2b4872c5460c27518c365a8ba03fc622d3354806" exitCode=0 Jul 16 09:11:27.545338 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.545333 2567 generic.go:356] "Generic (PLEG): container finished" podID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerID="51de7aaa5f4bd4a71f024966413b973aa38d50ef1ebfa95c288c7d2ff8aa9112" exitCode=0 Jul 16 09:11:27.545338 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.545339 2567 generic.go:356] "Generic (PLEG): container finished" podID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerID="e541dd66102231b2e7de21eef1c19480f46a2e3d3af37aa2868b46687e29e78f" exitCode=0 Jul 16 09:11:27.545338 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.545347 2567 generic.go:356] "Generic (PLEG): container finished" podID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerID="ade868a9b20cb4ce6db89c0deaf47bef0af78ccb54d19c15f13d8d84d60206ad" exitCode=0 Jul 16 09:11:27.545800 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.545355 2567 generic.go:356] "Generic (PLEG): container finished" podID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerID="717dc7583bc215b1ac8413a8fbe2f77a36b6a9ab80dedaa1c972b20bc56e4670" exitCode=0 Jul 16 09:11:27.545800 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.545373 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerDied","Data":"5c09336cd59eca33792e3abc2b4872c5460c27518c365a8ba03fc622d3354806"} Jul 16 09:11:27.545800 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.545438 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerDied","Data":"51de7aaa5f4bd4a71f024966413b973aa38d50ef1ebfa95c288c7d2ff8aa9112"} Jul 16 09:11:27.545800 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.545464 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerDied","Data":"e541dd66102231b2e7de21eef1c19480f46a2e3d3af37aa2868b46687e29e78f"} Jul 16 09:11:27.545800 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.545488 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerDied","Data":"ade868a9b20cb4ce6db89c0deaf47bef0af78ccb54d19c15f13d8d84d60206ad"} Jul 16 09:11:27.545800 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.545510 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerDied","Data":"717dc7583bc215b1ac8413a8fbe2f77a36b6a9ab80dedaa1c972b20bc56e4670"} Jul 16 09:11:27.747026 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.746996 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:27.855425 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.855340 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d00dd833-e80c-4da1-92d1-c169c10309a4-alertmanager-trusted-ca-bundle\") pod \"d00dd833-e80c-4da1-92d1-c169c10309a4\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " Jul 16 09:11:27.855425 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.855386 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d00dd833-e80c-4da1-92d1-c169c10309a4-tls-assets\") pod \"d00dd833-e80c-4da1-92d1-c169c10309a4\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " Jul 16 09:11:27.855425 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.855409 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-cluster-tls-config\") pod \"d00dd833-e80c-4da1-92d1-c169c10309a4\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " Jul 16 09:11:27.855699 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.855435 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy-web\") pod \"d00dd833-e80c-4da1-92d1-c169c10309a4\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " Jul 16 09:11:27.855699 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.855455 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f7jg\" (UniqueName: \"kubernetes.io/projected/d00dd833-e80c-4da1-92d1-c169c10309a4-kube-api-access-7f7jg\") pod \"d00dd833-e80c-4da1-92d1-c169c10309a4\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " Jul 16 09:11:27.855699 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.855483 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d00dd833-e80c-4da1-92d1-c169c10309a4-config-out\") pod \"d00dd833-e80c-4da1-92d1-c169c10309a4\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " Jul 16 09:11:27.855699 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.855518 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy\") pod \"d00dd833-e80c-4da1-92d1-c169c10309a4\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " Jul 16 09:11:27.855699 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.855549 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-main-tls\") pod \"d00dd833-e80c-4da1-92d1-c169c10309a4\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " Jul 16 09:11:27.855699 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.855590 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d00dd833-e80c-4da1-92d1-c169c10309a4-metrics-client-ca\") pod \"d00dd833-e80c-4da1-92d1-c169c10309a4\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " Jul 16 09:11:27.855699 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.855626 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d00dd833-e80c-4da1-92d1-c169c10309a4-alertmanager-main-db\") pod \"d00dd833-e80c-4da1-92d1-c169c10309a4\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " Jul 16 09:11:27.855699 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.855652 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d00dd833-e80c-4da1-92d1-c169c10309a4\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " Jul 16 09:11:27.855699 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.855685 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-web-config\") pod \"d00dd833-e80c-4da1-92d1-c169c10309a4\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " Jul 16 09:11:27.856173 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.855735 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-config-volume\") pod \"d00dd833-e80c-4da1-92d1-c169c10309a4\" (UID: \"d00dd833-e80c-4da1-92d1-c169c10309a4\") " Jul 16 09:11:27.856173 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.855803 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d00dd833-e80c-4da1-92d1-c169c10309a4-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d00dd833-e80c-4da1-92d1-c169c10309a4" (UID: "d00dd833-e80c-4da1-92d1-c169c10309a4"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:11:27.856173 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.856084 2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d00dd833-e80c-4da1-92d1-c169c10309a4-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:27.857023 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.856990 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d00dd833-e80c-4da1-92d1-c169c10309a4-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d00dd833-e80c-4da1-92d1-c169c10309a4" (UID: "d00dd833-e80c-4da1-92d1-c169c10309a4"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:11:27.857198 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.857113 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d00dd833-e80c-4da1-92d1-c169c10309a4-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d00dd833-e80c-4da1-92d1-c169c10309a4" (UID: "d00dd833-e80c-4da1-92d1-c169c10309a4"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 09:11:27.858323 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.858281 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d00dd833-e80c-4da1-92d1-c169c10309a4-config-out" (OuterVolumeSpecName: "config-out") pod "d00dd833-e80c-4da1-92d1-c169c10309a4" (UID: "d00dd833-e80c-4da1-92d1-c169c10309a4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 09:11:27.859357 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.859041 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d00dd833-e80c-4da1-92d1-c169c10309a4" (UID: "d00dd833-e80c-4da1-92d1-c169c10309a4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:27.859357 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.859132 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-config-volume" (OuterVolumeSpecName: "config-volume") pod "d00dd833-e80c-4da1-92d1-c169c10309a4" (UID: "d00dd833-e80c-4da1-92d1-c169c10309a4"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:27.859932 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.859895 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d00dd833-e80c-4da1-92d1-c169c10309a4" (UID: "d00dd833-e80c-4da1-92d1-c169c10309a4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:27.859932 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.859902 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d00dd833-e80c-4da1-92d1-c169c10309a4-kube-api-access-7f7jg" (OuterVolumeSpecName: "kube-api-access-7f7jg") pod "d00dd833-e80c-4da1-92d1-c169c10309a4" (UID: "d00dd833-e80c-4da1-92d1-c169c10309a4"). InnerVolumeSpecName "kube-api-access-7f7jg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 09:11:27.860276 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.860248 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d00dd833-e80c-4da1-92d1-c169c10309a4" (UID: "d00dd833-e80c-4da1-92d1-c169c10309a4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:27.860692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.860671 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d00dd833-e80c-4da1-92d1-c169c10309a4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d00dd833-e80c-4da1-92d1-c169c10309a4" (UID: "d00dd833-e80c-4da1-92d1-c169c10309a4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 09:11:27.860775 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.860763 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d00dd833-e80c-4da1-92d1-c169c10309a4" (UID: "d00dd833-e80c-4da1-92d1-c169c10309a4"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:27.863431 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.863410 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d00dd833-e80c-4da1-92d1-c169c10309a4" (UID: "d00dd833-e80c-4da1-92d1-c169c10309a4"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:27.871165 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.871141 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-web-config" (OuterVolumeSpecName: "web-config") pod "d00dd833-e80c-4da1-92d1-c169c10309a4" (UID: "d00dd833-e80c-4da1-92d1-c169c10309a4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:27.956603 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.956564 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-main-tls\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:27.956603 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.956598 2567 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d00dd833-e80c-4da1-92d1-c169c10309a4-metrics-client-ca\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:27.956603 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.956609 2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d00dd833-e80c-4da1-92d1-c169c10309a4-alertmanager-main-db\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:27.956863 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.956619 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:27.956863 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.956629 2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-web-config\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:27.956863 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.956638 2567 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-config-volume\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:27.956863 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.956647 2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d00dd833-e80c-4da1-92d1-c169c10309a4-tls-assets\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:27.956863 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.956655 2567 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-cluster-tls-config\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:27.956863 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.956663 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:27.956863 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.956673 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f7jg\" (UniqueName: \"kubernetes.io/projected/d00dd833-e80c-4da1-92d1-c169c10309a4-kube-api-access-7f7jg\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:27.956863 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.956681 2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d00dd833-e80c-4da1-92d1-c169c10309a4-config-out\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:27.956863 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:27.956689 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d00dd833-e80c-4da1-92d1-c169c10309a4-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:28.552745 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.552709 2567 generic.go:356] "Generic (PLEG): container finished" podID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerID="d073ba0a71a7ea8dc40a76382e4f09217a8034ea4229208a0cdee3bb2a50f455" exitCode=0 Jul 16 09:11:28.552745 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.552746 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerDied","Data":"d073ba0a71a7ea8dc40a76382e4f09217a8034ea4229208a0cdee3bb2a50f455"} Jul 16 09:11:28.553185 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.552782 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.553185 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.552808 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d00dd833-e80c-4da1-92d1-c169c10309a4","Type":"ContainerDied","Data":"e71a3378189da3c435b27c1208537c1d7801d33f9ac0bebf59c5933e6bcddacd"} Jul 16 09:11:28.553185 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.552852 2567 scope.go:117] "RemoveContainer" containerID="5c09336cd59eca33792e3abc2b4872c5460c27518c365a8ba03fc622d3354806" Jul 16 09:11:28.560801 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.560783 2567 scope.go:117] "RemoveContainer" containerID="51de7aaa5f4bd4a71f024966413b973aa38d50ef1ebfa95c288c7d2ff8aa9112" Jul 16 09:11:28.568095 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.568072 2567 scope.go:117] "RemoveContainer" containerID="e541dd66102231b2e7de21eef1c19480f46a2e3d3af37aa2868b46687e29e78f" Jul 16 09:11:28.575211 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.575190 2567 scope.go:117] "RemoveContainer" containerID="d073ba0a71a7ea8dc40a76382e4f09217a8034ea4229208a0cdee3bb2a50f455" Jul 16 09:11:28.581281 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.581261 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 09:11:28.583234 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.583181 2567 scope.go:117] "RemoveContainer" containerID="ade868a9b20cb4ce6db89c0deaf47bef0af78ccb54d19c15f13d8d84d60206ad" Jul 16 09:11:28.584946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.584929 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 09:11:28.590336 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.590309 2567 scope.go:117] "RemoveContainer" containerID="717dc7583bc215b1ac8413a8fbe2f77a36b6a9ab80dedaa1c972b20bc56e4670" Jul 16 09:11:28.599208 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.599186 2567 scope.go:117] "RemoveContainer" containerID="4835322c040cf7813dc22846944818a7af2e6c5b42cee2b872391d60c200ed57" Jul 16 09:11:28.606310 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.606295 2567 scope.go:117] "RemoveContainer" containerID="5c09336cd59eca33792e3abc2b4872c5460c27518c365a8ba03fc622d3354806" Jul 16 09:11:28.606541 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:11:28.606524 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c09336cd59eca33792e3abc2b4872c5460c27518c365a8ba03fc622d3354806\": container with ID starting with 5c09336cd59eca33792e3abc2b4872c5460c27518c365a8ba03fc622d3354806 not found: ID does not exist" containerID="5c09336cd59eca33792e3abc2b4872c5460c27518c365a8ba03fc622d3354806" Jul 16 09:11:28.606599 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.606549 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c09336cd59eca33792e3abc2b4872c5460c27518c365a8ba03fc622d3354806"} err="failed to get container status \"5c09336cd59eca33792e3abc2b4872c5460c27518c365a8ba03fc622d3354806\": rpc error: code = NotFound desc = could not find container \"5c09336cd59eca33792e3abc2b4872c5460c27518c365a8ba03fc622d3354806\": container with ID starting with 5c09336cd59eca33792e3abc2b4872c5460c27518c365a8ba03fc622d3354806 not found: ID does not exist" Jul 16 09:11:28.606599 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.606566 2567 scope.go:117] "RemoveContainer" containerID="51de7aaa5f4bd4a71f024966413b973aa38d50ef1ebfa95c288c7d2ff8aa9112" Jul 16 09:11:28.611965 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:11:28.611923 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51de7aaa5f4bd4a71f024966413b973aa38d50ef1ebfa95c288c7d2ff8aa9112\": container with ID starting with 51de7aaa5f4bd4a71f024966413b973aa38d50ef1ebfa95c288c7d2ff8aa9112 not found: ID does not exist" containerID="51de7aaa5f4bd4a71f024966413b973aa38d50ef1ebfa95c288c7d2ff8aa9112" Jul 16 09:11:28.612531 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.612256 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51de7aaa5f4bd4a71f024966413b973aa38d50ef1ebfa95c288c7d2ff8aa9112"} err="failed to get container status \"51de7aaa5f4bd4a71f024966413b973aa38d50ef1ebfa95c288c7d2ff8aa9112\": rpc error: code = NotFound desc = could not find container \"51de7aaa5f4bd4a71f024966413b973aa38d50ef1ebfa95c288c7d2ff8aa9112\": container with ID starting with 51de7aaa5f4bd4a71f024966413b973aa38d50ef1ebfa95c288c7d2ff8aa9112 not found: ID does not exist" Jul 16 09:11:28.612636 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.612535 2567 scope.go:117] "RemoveContainer" containerID="e541dd66102231b2e7de21eef1c19480f46a2e3d3af37aa2868b46687e29e78f" Jul 16 09:11:28.612860 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:11:28.612841 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e541dd66102231b2e7de21eef1c19480f46a2e3d3af37aa2868b46687e29e78f\": container with ID starting with e541dd66102231b2e7de21eef1c19480f46a2e3d3af37aa2868b46687e29e78f not found: ID does not exist" containerID="e541dd66102231b2e7de21eef1c19480f46a2e3d3af37aa2868b46687e29e78f" Jul 16 09:11:28.612923 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.612867 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e541dd66102231b2e7de21eef1c19480f46a2e3d3af37aa2868b46687e29e78f"} err="failed to get container status \"e541dd66102231b2e7de21eef1c19480f46a2e3d3af37aa2868b46687e29e78f\": rpc error: code = NotFound desc = could not find container \"e541dd66102231b2e7de21eef1c19480f46a2e3d3af37aa2868b46687e29e78f\": container with ID starting with e541dd66102231b2e7de21eef1c19480f46a2e3d3af37aa2868b46687e29e78f not found: ID does not exist" Jul 16 09:11:28.612923 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.612882 2567 scope.go:117] "RemoveContainer" containerID="d073ba0a71a7ea8dc40a76382e4f09217a8034ea4229208a0cdee3bb2a50f455" Jul 16 09:11:28.613129 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:11:28.613113 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d073ba0a71a7ea8dc40a76382e4f09217a8034ea4229208a0cdee3bb2a50f455\": container with ID starting with d073ba0a71a7ea8dc40a76382e4f09217a8034ea4229208a0cdee3bb2a50f455 not found: ID does not exist" containerID="d073ba0a71a7ea8dc40a76382e4f09217a8034ea4229208a0cdee3bb2a50f455" Jul 16 09:11:28.613180 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.613133 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d073ba0a71a7ea8dc40a76382e4f09217a8034ea4229208a0cdee3bb2a50f455"} err="failed to get container status \"d073ba0a71a7ea8dc40a76382e4f09217a8034ea4229208a0cdee3bb2a50f455\": rpc error: code = NotFound desc = could not find container \"d073ba0a71a7ea8dc40a76382e4f09217a8034ea4229208a0cdee3bb2a50f455\": container with ID starting with d073ba0a71a7ea8dc40a76382e4f09217a8034ea4229208a0cdee3bb2a50f455 not found: ID does not exist" Jul 16 09:11:28.613180 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.613145 2567 scope.go:117] "RemoveContainer" containerID="ade868a9b20cb4ce6db89c0deaf47bef0af78ccb54d19c15f13d8d84d60206ad" Jul 16 09:11:28.613365 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:11:28.613348 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ade868a9b20cb4ce6db89c0deaf47bef0af78ccb54d19c15f13d8d84d60206ad\": container with ID starting with ade868a9b20cb4ce6db89c0deaf47bef0af78ccb54d19c15f13d8d84d60206ad not found: ID does not exist" containerID="ade868a9b20cb4ce6db89c0deaf47bef0af78ccb54d19c15f13d8d84d60206ad" Jul 16 09:11:28.613435 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.613373 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade868a9b20cb4ce6db89c0deaf47bef0af78ccb54d19c15f13d8d84d60206ad"} err="failed to get container status \"ade868a9b20cb4ce6db89c0deaf47bef0af78ccb54d19c15f13d8d84d60206ad\": rpc error: code = NotFound desc = could not find container \"ade868a9b20cb4ce6db89c0deaf47bef0af78ccb54d19c15f13d8d84d60206ad\": container with ID starting with ade868a9b20cb4ce6db89c0deaf47bef0af78ccb54d19c15f13d8d84d60206ad not found: ID does not exist" Jul 16 09:11:28.613435 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.613394 2567 scope.go:117] "RemoveContainer" containerID="717dc7583bc215b1ac8413a8fbe2f77a36b6a9ab80dedaa1c972b20bc56e4670" Jul 16 09:11:28.613647 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:11:28.613631 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"717dc7583bc215b1ac8413a8fbe2f77a36b6a9ab80dedaa1c972b20bc56e4670\": container with ID starting with 717dc7583bc215b1ac8413a8fbe2f77a36b6a9ab80dedaa1c972b20bc56e4670 not found: ID does not exist" containerID="717dc7583bc215b1ac8413a8fbe2f77a36b6a9ab80dedaa1c972b20bc56e4670" Jul 16 09:11:28.613685 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.613650 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"717dc7583bc215b1ac8413a8fbe2f77a36b6a9ab80dedaa1c972b20bc56e4670"} err="failed to get container status \"717dc7583bc215b1ac8413a8fbe2f77a36b6a9ab80dedaa1c972b20bc56e4670\": rpc error: code = NotFound desc = could not find container \"717dc7583bc215b1ac8413a8fbe2f77a36b6a9ab80dedaa1c972b20bc56e4670\": container with ID starting with 717dc7583bc215b1ac8413a8fbe2f77a36b6a9ab80dedaa1c972b20bc56e4670 not found: ID does not exist" Jul 16 09:11:28.613685 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.613663 2567 scope.go:117] "RemoveContainer" containerID="4835322c040cf7813dc22846944818a7af2e6c5b42cee2b872391d60c200ed57" Jul 16 09:11:28.613954 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:11:28.613936 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4835322c040cf7813dc22846944818a7af2e6c5b42cee2b872391d60c200ed57\": container with ID starting with 4835322c040cf7813dc22846944818a7af2e6c5b42cee2b872391d60c200ed57 not found: ID does not exist" containerID="4835322c040cf7813dc22846944818a7af2e6c5b42cee2b872391d60c200ed57" Jul 16 09:11:28.614012 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.613962 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4835322c040cf7813dc22846944818a7af2e6c5b42cee2b872391d60c200ed57"} err="failed to get container status \"4835322c040cf7813dc22846944818a7af2e6c5b42cee2b872391d60c200ed57\": rpc error: code = NotFound desc = could not find container \"4835322c040cf7813dc22846944818a7af2e6c5b42cee2b872391d60c200ed57\": container with ID starting with 4835322c040cf7813dc22846944818a7af2e6c5b42cee2b872391d60c200ed57 not found: ID does not exist" Jul 16 09:11:28.614384 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614368 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 09:11:28.614673 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614662 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="kube-rbac-proxy-web" Jul 16 09:11:28.614708 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614676 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="kube-rbac-proxy-web" Jul 16 09:11:28.614708 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614684 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="prom-label-proxy" Jul 16 09:11:28.614708 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614690 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="prom-label-proxy" Jul 16 09:11:28.614708 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614700 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="init-config-reloader" Jul 16 09:11:28.614708 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614705 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="init-config-reloader" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614718 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="kube-rbac-proxy" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614723 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="kube-rbac-proxy" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614732 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="alertmanager" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614738 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="alertmanager" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614745 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="kube-rbac-proxy-metric" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614751 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="kube-rbac-proxy-metric" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614759 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4cf179b-d79d-44c6-a486-6cc872cc84ff" containerName="registry" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614764 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4cf179b-d79d-44c6-a486-6cc872cc84ff" containerName="registry" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614771 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff918822-58d7-49cf-a8a5-c860e147af23" containerName="console" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614776 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff918822-58d7-49cf-a8a5-c860e147af23" containerName="console" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614782 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="config-reloader" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614787 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="config-reloader" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614850 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4cf179b-d79d-44c6-a486-6cc872cc84ff" containerName="registry" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614858 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="config-reloader" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614865 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="kube-rbac-proxy-metric" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614872 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="kube-rbac-proxy-web" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614878 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff918822-58d7-49cf-a8a5-c860e147af23" containerName="console" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614884 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="kube-rbac-proxy" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614889 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="alertmanager" Jul 16 09:11:28.614903 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.614894 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" containerName="prom-label-proxy" Jul 16 09:11:28.618586 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.618571 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.621526 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.621382 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 16 09:11:28.621526 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.621395 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 16 09:11:28.621526 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.621414 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 16 09:11:28.621526 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.621382 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h7z9s\"" Jul 16 09:11:28.621526 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.621388 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 16 09:11:28.621526 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.621382 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 16 09:11:28.621526 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.621525 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 16 09:11:28.621919 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.621700 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 16 09:11:28.621919 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.621723 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 16 09:11:28.627679 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.627662 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 16 09:11:28.631440 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.631421 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 09:11:28.763422 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.763393 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-web-config\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.763581 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.763428 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.763581 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.763451 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-config-out\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.763581 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.763472 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-config-volume\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.763581 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.763532 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.763581 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.763565 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.763846 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.763593 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.763846 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.763608 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.763846 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.763680 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-tls-assets\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.763846 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.763719 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.763846 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.763775 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.763846 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.763802 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nc9v\" (UniqueName: \"kubernetes.io/projected/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-kube-api-access-7nc9v\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.764102 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.763859 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.864283 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.864213 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.864283 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.864252 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nc9v\" (UniqueName: \"kubernetes.io/projected/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-kube-api-access-7nc9v\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.864283 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.864275 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.864532 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.864315 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-web-config\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.864532 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.864332 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.864532 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.864352 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-config-out\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.864532 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.864389 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-config-volume\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.864532 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.864424 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.864532 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.864451 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.864532 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.864486 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.864532 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.864509 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.864948 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.864552 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-tls-assets\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.864948 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.864574 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.865473 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.865446 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.867775 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.867510 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.867775 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.867596 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-config-volume\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.867775 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.867680 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-web-config\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.867775 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.867713 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.868063 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.867789 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.868063 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.867882 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.868063 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.868035 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-config-out\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.868174 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.868124 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.868242 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.868224 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.868610 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.868594 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.869536 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.869516 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-tls-assets\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.872898 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.872877 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nc9v\" (UniqueName: \"kubernetes.io/projected/72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71-kube-api-access-7nc9v\") pod \"alertmanager-main-0\" (UID: \"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:28.928556 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:28.928533 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 09:11:29.081219 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:29.081197 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 09:11:29.082804 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:11:29.082778 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72abe2c8_0ca6_4c59_8cd5_45f1d7eb5a71.slice/crio-ba71f45e537343289262a2eda90ac2f0495444d10b6bf5e5dfba92e4fc7ea80c WatchSource:0}: Error finding container ba71f45e537343289262a2eda90ac2f0495444d10b6bf5e5dfba92e4fc7ea80c: Status 404 returned error can't find the container with id ba71f45e537343289262a2eda90ac2f0495444d10b6bf5e5dfba92e4fc7ea80c Jul 16 09:11:29.323510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:29.323474 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d00dd833-e80c-4da1-92d1-c169c10309a4" path="/var/lib/kubelet/pods/d00dd833-e80c-4da1-92d1-c169c10309a4/volumes" Jul 16 09:11:29.558779 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:29.558745 2567 generic.go:356] "Generic (PLEG): container finished" podID="72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71" containerID="1a0ffeeeda506be1162f0112ba5ca2c9f7ed7961a99c9d8c5cd0d5c262ef2494" exitCode=0 Jul 16 09:11:29.559232 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:29.558782 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71","Type":"ContainerDied","Data":"1a0ffeeeda506be1162f0112ba5ca2c9f7ed7961a99c9d8c5cd0d5c262ef2494"} Jul 16 09:11:29.559232 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:29.558864 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71","Type":"ContainerStarted","Data":"ba71f45e537343289262a2eda90ac2f0495444d10b6bf5e5dfba92e4fc7ea80c"} Jul 16 09:11:30.566788 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:30.566751 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71","Type":"ContainerStarted","Data":"451e96a2198eef4509885d13150da90c8d2ea55f2d23fcb1750e2d64e65a0458"} Jul 16 09:11:30.567265 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:30.566853 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71","Type":"ContainerStarted","Data":"d08b3112eb78ee03f00c7182c7aac2cb975bfe4b378e23dbd884e7a84a05fee7"} Jul 16 09:11:30.567265 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:30.566898 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71","Type":"ContainerStarted","Data":"749b27391f9f0f2e33aa004a5a2b545e38954fb0faa0fb271f59fa633723c569"} Jul 16 09:11:30.567265 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:30.566944 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71","Type":"ContainerStarted","Data":"c8990302cfa9dd9b5589f4a73f6f02019eb30d8dd7d11eec114ee5a99cf806b3"} Jul 16 09:11:30.567265 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:30.566985 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71","Type":"ContainerStarted","Data":"292a92f70df90a8a6b16685ba85951c80e0d8a003c6328a5d12903c0dff0c558"} Jul 16 09:11:30.567265 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:30.567037 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71","Type":"ContainerStarted","Data":"a8378520650fc5160a99b02230b7a8150dc9fd231adee8780c4d16371a4460d7"} Jul 16 09:11:30.601859 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:30.601789 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.586269417 podStartE2EDuration="2.60177516s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.505743ms" isStatefulPod=false podCreationTimestamp="2026-07-16 09:11:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:11:30.59919752 +0000 UTC m=+366.008745779" watchObservedRunningTime="2026-07-16 09:11:30.60177516 +0000 UTC m=+366.011323418" Jul 16 09:11:30.838904 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:30.838804 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 09:11:30.839105 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:30.839082 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="prometheus" containerID="cri-o://4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30" gracePeriod=600 Jul 16 09:11:30.839169 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:30.839111 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="kube-rbac-proxy" containerID="cri-o://26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d" gracePeriod=600 Jul 16 09:11:30.839169 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:30.839131 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="config-reloader" containerID="cri-o://46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50" gracePeriod=600 Jul 16 09:11:30.839169 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:30.839154 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="kube-rbac-proxy-web" containerID="cri-o://5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c" gracePeriod=600 Jul 16 09:11:30.839359 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:30.839158 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e" gracePeriod=600 Jul 16 09:11:30.839359 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:30.839117 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="thanos-sidecar" containerID="cri-o://450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6" gracePeriod=600 Jul 16 09:11:31.069837 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.069797 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.185857 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.185750 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-db\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.185857 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.185804 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-web-config\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.185857 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.185844 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-metrics-client-certs\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.185870 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w5ns\" (UniqueName: \"kubernetes.io/projected/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-kube-api-access-9w5ns\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.185899 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-1\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.185915 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-trusted-ca-bundle\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186030 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186082 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-grpc-tls\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186124 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-thanos-prometheus-http-client-file\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186155 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-tls-assets\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186209 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-kube-rbac-proxy\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186239 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-serving-certs-ca-bundle\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186268 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-tls\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186268 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:11:31.186420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186290 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:11:31.186420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186302 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186327 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-0\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186355 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-config\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186387 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-metrics-client-ca\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.186420 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186410 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-2\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.189276 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186435 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-config-out\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.189276 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186464 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-kubelet-serving-ca-bundle\") pod \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\" (UID: \"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c\") " Jul 16 09:11:31.189276 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.186654 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:11:31.189276 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.187124 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:11:31.189276 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.187220 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.189276 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.187242 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.189276 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.187259 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.189276 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.187260 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 09:11:31.189276 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.187520 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:11:31.189276 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.187851 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:11:31.189276 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.188881 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:11:31.189276 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.189225 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:31.189913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.189302 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-kube-api-access-9w5ns" (OuterVolumeSpecName: "kube-api-access-9w5ns") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "kube-api-access-9w5ns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 09:11:31.190016 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.189964 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:31.190132 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.190107 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:31.190212 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.190187 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-config-out" (OuterVolumeSpecName: "config-out") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 09:11:31.190365 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.190333 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 09:11:31.190593 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.190568 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:31.190713 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.190639 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:31.191368 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.191346 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:31.191460 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.191378 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:31.191666 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.191643 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-config" (OuterVolumeSpecName: "config") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:31.200171 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.200152 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-web-config" (OuterVolumeSpecName: "web-config") pod "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" (UID: "a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:11:31.287953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.287922 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.287953 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.287949 2567 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-grpc-tls\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.287960 2567 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.287970 2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-tls-assets\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.287979 2567 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-kube-rbac-proxy\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.287988 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.287997 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.288005 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.288015 2567 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-config\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.288023 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-metrics-client-ca\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.288032 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.288040 2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-config-out\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.288049 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.288056 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-prometheus-k8s-db\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.288065 2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-web-config\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.288073 2567 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-secret-metrics-client-certs\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.288128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.288081 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w5ns\" (UniqueName: \"kubernetes.io/projected/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c-kube-api-access-9w5ns\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:11:31.574333 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574299 2567 generic.go:356] "Generic (PLEG): container finished" podID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerID="f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e" exitCode=0 Jul 16 09:11:31.574333 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574325 2567 generic.go:356] "Generic (PLEG): container finished" podID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerID="26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d" exitCode=0 Jul 16 09:11:31.574333 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574332 2567 generic.go:356] "Generic (PLEG): container finished" podID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerID="5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c" exitCode=0 Jul 16 09:11:31.574333 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574338 2567 generic.go:356] "Generic (PLEG): container finished" podID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerID="450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6" exitCode=0 Jul 16 09:11:31.574843 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574343 2567 generic.go:356] "Generic (PLEG): container finished" podID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerID="46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50" exitCode=0 Jul 16 09:11:31.574843 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574350 2567 generic.go:356] "Generic (PLEG): container finished" podID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerID="4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30" exitCode=0 Jul 16 09:11:31.574843 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574373 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerDied","Data":"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e"} Jul 16 09:11:31.574843 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574445 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerDied","Data":"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d"} Jul 16 09:11:31.574843 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574387 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.574843 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574474 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerDied","Data":"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c"} Jul 16 09:11:31.574843 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574501 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerDied","Data":"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6"} Jul 16 09:11:31.574843 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574525 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerDied","Data":"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50"} Jul 16 09:11:31.574843 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574540 2567 scope.go:117] "RemoveContainer" containerID="f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e" Jul 16 09:11:31.574843 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574548 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerDied","Data":"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30"} Jul 16 09:11:31.574843 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.574573 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c","Type":"ContainerDied","Data":"2b72ea653a145fd43e7300922f45db4693aa30a96e42e889df5057b3f7007a37"} Jul 16 09:11:31.582583 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.582567 2567 scope.go:117] "RemoveContainer" containerID="26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d" Jul 16 09:11:31.589455 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.589434 2567 scope.go:117] "RemoveContainer" containerID="5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c" Jul 16 09:11:31.596252 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.596234 2567 scope.go:117] "RemoveContainer" containerID="450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6" Jul 16 09:11:31.606011 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.605991 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 09:11:31.614746 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.614684 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 09:11:31.614838 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.614806 2567 scope.go:117] "RemoveContainer" containerID="46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50" Jul 16 09:11:31.621810 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.621794 2567 scope.go:117] "RemoveContainer" containerID="4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30" Jul 16 09:11:31.629083 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.629064 2567 scope.go:117] "RemoveContainer" containerID="652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d" Jul 16 09:11:31.635629 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.635614 2567 scope.go:117] "RemoveContainer" containerID="f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e" Jul 16 09:11:31.635901 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:11:31.635882 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e\": container with ID starting with f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e not found: ID does not exist" containerID="f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e" Jul 16 09:11:31.635951 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.635912 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e"} err="failed to get container status \"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e\": rpc error: code = NotFound desc = could not find container \"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e\": container with ID starting with f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e not found: ID does not exist" Jul 16 09:11:31.635951 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.635931 2567 scope.go:117] "RemoveContainer" containerID="26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d" Jul 16 09:11:31.636177 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:11:31.636158 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d\": container with ID starting with 26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d not found: ID does not exist" containerID="26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d" Jul 16 09:11:31.636244 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.636187 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d"} err="failed to get container status \"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d\": rpc error: code = NotFound desc = could not find container \"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d\": container with ID starting with 26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d not found: ID does not exist" Jul 16 09:11:31.636244 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.636209 2567 scope.go:117] "RemoveContainer" containerID="5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c" Jul 16 09:11:31.636478 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:11:31.636461 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c\": container with ID starting with 5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c not found: ID does not exist" containerID="5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c" Jul 16 09:11:31.636518 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.636482 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c"} err="failed to get container status \"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c\": rpc error: code = NotFound desc = could not find container \"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c\": container with ID starting with 5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c not found: ID does not exist" Jul 16 09:11:31.636518 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.636497 2567 scope.go:117] "RemoveContainer" containerID="450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6" Jul 16 09:11:31.636703 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:11:31.636687 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6\": container with ID starting with 450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6 not found: ID does not exist" containerID="450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6" Jul 16 09:11:31.636768 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.636710 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6"} err="failed to get container status \"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6\": rpc error: code = NotFound desc = could not find container \"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6\": container with ID starting with 450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6 not found: ID does not exist" Jul 16 09:11:31.636768 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.636728 2567 scope.go:117] "RemoveContainer" containerID="46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50" Jul 16 09:11:31.636983 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:11:31.636967 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50\": container with ID starting with 46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50 not found: ID does not exist" containerID="46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50" Jul 16 09:11:31.637027 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.636986 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50"} err="failed to get container status \"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50\": rpc error: code = NotFound desc = could not find container \"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50\": container with ID starting with 46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50 not found: ID does not exist" Jul 16 09:11:31.637027 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.637001 2567 scope.go:117] "RemoveContainer" containerID="4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30" Jul 16 09:11:31.637222 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:11:31.637200 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30\": container with ID starting with 4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30 not found: ID does not exist" containerID="4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30" Jul 16 09:11:31.637273 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.637220 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30"} err="failed to get container status \"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30\": rpc error: code = NotFound desc = could not find container \"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30\": container with ID starting with 4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30 not found: ID does not exist" Jul 16 09:11:31.637273 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.637236 2567 scope.go:117] "RemoveContainer" containerID="652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d" Jul 16 09:11:31.637461 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:11:31.637444 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d\": container with ID starting with 652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d not found: ID does not exist" containerID="652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d" Jul 16 09:11:31.637522 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.637467 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d"} err="failed to get container status \"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d\": rpc error: code = NotFound desc = could not find container \"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d\": container with ID starting with 652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d not found: ID does not exist" Jul 16 09:11:31.637522 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.637486 2567 scope.go:117] "RemoveContainer" containerID="f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e" Jul 16 09:11:31.637700 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.637683 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e"} err="failed to get container status \"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e\": rpc error: code = NotFound desc = could not find container \"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e\": container with ID starting with f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e not found: ID does not exist" Jul 16 09:11:31.637759 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.637701 2567 scope.go:117] "RemoveContainer" containerID="26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d" Jul 16 09:11:31.637946 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.637925 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d"} err="failed to get container status \"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d\": rpc error: code = NotFound desc = could not find container \"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d\": container with ID starting with 26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d not found: ID does not exist" Jul 16 09:11:31.637999 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.637948 2567 scope.go:117] "RemoveContainer" containerID="5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c" Jul 16 09:11:31.638189 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.638172 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c"} err="failed to get container status \"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c\": rpc error: code = NotFound desc = could not find container \"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c\": container with ID starting with 5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c not found: ID does not exist" Jul 16 09:11:31.638240 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.638189 2567 scope.go:117] "RemoveContainer" containerID="450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6" Jul 16 09:11:31.638421 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.638404 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6"} err="failed to get container status \"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6\": rpc error: code = NotFound desc = could not find container \"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6\": container with ID starting with 450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6 not found: ID does not exist" Jul 16 09:11:31.638481 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.638422 2567 scope.go:117] "RemoveContainer" containerID="46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50" Jul 16 09:11:31.638598 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.638582 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50"} err="failed to get container status \"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50\": rpc error: code = NotFound desc = could not find container \"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50\": container with ID starting with 46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50 not found: ID does not exist" Jul 16 09:11:31.638635 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.638600 2567 scope.go:117] "RemoveContainer" containerID="4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30" Jul 16 09:11:31.638775 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.638758 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30"} err="failed to get container status \"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30\": rpc error: code = NotFound desc = could not find container \"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30\": container with ID starting with 4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30 not found: ID does not exist" Jul 16 09:11:31.638831 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.638776 2567 scope.go:117] "RemoveContainer" containerID="652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d" Jul 16 09:11:31.639000 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.638979 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d"} err="failed to get container status \"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d\": rpc error: code = NotFound desc = could not find container \"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d\": container with ID starting with 652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d not found: ID does not exist" Jul 16 09:11:31.639000 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.638995 2567 scope.go:117] "RemoveContainer" containerID="f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e" Jul 16 09:11:31.639211 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.639196 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e"} err="failed to get container status \"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e\": rpc error: code = NotFound desc = could not find container \"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e\": container with ID starting with f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e not found: ID does not exist" Jul 16 09:11:31.639260 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.639212 2567 scope.go:117] "RemoveContainer" containerID="26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d" Jul 16 09:11:31.639407 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.639391 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d"} err="failed to get container status \"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d\": rpc error: code = NotFound desc = could not find container \"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d\": container with ID starting with 26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d not found: ID does not exist" Jul 16 09:11:31.639455 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.639407 2567 scope.go:117] "RemoveContainer" containerID="5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c" Jul 16 09:11:31.639560 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.639545 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c"} err="failed to get container status \"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c\": rpc error: code = NotFound desc = could not find container \"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c\": container with ID starting with 5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c not found: ID does not exist" Jul 16 09:11:31.639605 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.639560 2567 scope.go:117] "RemoveContainer" containerID="450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6" Jul 16 09:11:31.639785 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.639758 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6"} err="failed to get container status \"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6\": rpc error: code = NotFound desc = could not find container \"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6\": container with ID starting with 450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6 not found: ID does not exist" Jul 16 09:11:31.639785 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.639784 2567 scope.go:117] "RemoveContainer" containerID="46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50" Jul 16 09:11:31.640042 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.640021 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50"} err="failed to get container status \"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50\": rpc error: code = NotFound desc = could not find container \"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50\": container with ID starting with 46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50 not found: ID does not exist" Jul 16 09:11:31.640096 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.640043 2567 scope.go:117] "RemoveContainer" containerID="4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30" Jul 16 09:11:31.640309 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.640246 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30"} err="failed to get container status \"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30\": rpc error: code = NotFound desc = could not find container \"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30\": container with ID starting with 4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30 not found: ID does not exist" Jul 16 09:11:31.640309 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.640269 2567 scope.go:117] "RemoveContainer" containerID="652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d" Jul 16 09:11:31.640502 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.640484 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d"} err="failed to get container status \"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d\": rpc error: code = NotFound desc = could not find container \"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d\": container with ID starting with 652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d not found: ID does not exist" Jul 16 09:11:31.640544 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.640502 2567 scope.go:117] "RemoveContainer" containerID="f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e" Jul 16 09:11:31.640708 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.640692 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e"} err="failed to get container status \"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e\": rpc error: code = NotFound desc = could not find container \"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e\": container with ID starting with f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e not found: ID does not exist" Jul 16 09:11:31.640752 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.640709 2567 scope.go:117] "RemoveContainer" containerID="26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d" Jul 16 09:11:31.640924 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.640909 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d"} err="failed to get container status \"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d\": rpc error: code = NotFound desc = could not find container \"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d\": container with ID starting with 26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d not found: ID does not exist" Jul 16 09:11:31.640962 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.640925 2567 scope.go:117] "RemoveContainer" containerID="5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c" Jul 16 09:11:31.641138 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.641122 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c"} err="failed to get container status \"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c\": rpc error: code = NotFound desc = could not find container \"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c\": container with ID starting with 5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c not found: ID does not exist" Jul 16 09:11:31.641182 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.641138 2567 scope.go:117] "RemoveContainer" containerID="450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6" Jul 16 09:11:31.641386 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.641368 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6"} err="failed to get container status \"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6\": rpc error: code = NotFound desc = could not find container \"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6\": container with ID starting with 450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6 not found: ID does not exist" Jul 16 09:11:31.641425 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.641388 2567 scope.go:117] "RemoveContainer" containerID="46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50" Jul 16 09:11:31.641591 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.641575 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50"} err="failed to get container status \"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50\": rpc error: code = NotFound desc = could not find container \"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50\": container with ID starting with 46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50 not found: ID does not exist" Jul 16 09:11:31.641631 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.641592 2567 scope.go:117] "RemoveContainer" containerID="4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30" Jul 16 09:11:31.641808 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.641791 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30"} err="failed to get container status \"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30\": rpc error: code = NotFound desc = could not find container \"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30\": container with ID starting with 4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30 not found: ID does not exist" Jul 16 09:11:31.641877 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.641808 2567 scope.go:117] "RemoveContainer" containerID="652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d" Jul 16 09:11:31.642062 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.642044 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d"} err="failed to get container status \"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d\": rpc error: code = NotFound desc = could not find container \"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d\": container with ID starting with 652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d not found: ID does not exist" Jul 16 09:11:31.642097 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.642063 2567 scope.go:117] "RemoveContainer" containerID="f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e" Jul 16 09:11:31.642271 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.642256 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e"} err="failed to get container status \"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e\": rpc error: code = NotFound desc = could not find container \"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e\": container with ID starting with f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e not found: ID does not exist" Jul 16 09:11:31.642308 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.642272 2567 scope.go:117] "RemoveContainer" containerID="26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d" Jul 16 09:11:31.642496 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.642473 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d"} err="failed to get container status \"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d\": rpc error: code = NotFound desc = could not find container \"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d\": container with ID starting with 26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d not found: ID does not exist" Jul 16 09:11:31.642574 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.642498 2567 scope.go:117] "RemoveContainer" containerID="5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c" Jul 16 09:11:31.642687 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.642669 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c"} err="failed to get container status \"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c\": rpc error: code = NotFound desc = could not find container \"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c\": container with ID starting with 5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c not found: ID does not exist" Jul 16 09:11:31.642728 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.642688 2567 scope.go:117] "RemoveContainer" containerID="450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6" Jul 16 09:11:31.642929 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.642912 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6"} err="failed to get container status \"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6\": rpc error: code = NotFound desc = could not find container \"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6\": container with ID starting with 450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6 not found: ID does not exist" Jul 16 09:11:31.642979 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.642929 2567 scope.go:117] "RemoveContainer" containerID="46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50" Jul 16 09:11:31.643136 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.643120 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50"} err="failed to get container status \"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50\": rpc error: code = NotFound desc = could not find container \"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50\": container with ID starting with 46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50 not found: ID does not exist" Jul 16 09:11:31.643186 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.643136 2567 scope.go:117] "RemoveContainer" containerID="4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30" Jul 16 09:11:31.643329 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.643312 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30"} err="failed to get container status \"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30\": rpc error: code = NotFound desc = could not find container \"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30\": container with ID starting with 4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30 not found: ID does not exist" Jul 16 09:11:31.643374 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.643329 2567 scope.go:117] "RemoveContainer" containerID="652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d" Jul 16 09:11:31.643558 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.643543 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d"} err="failed to get container status \"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d\": rpc error: code = NotFound desc = could not find container \"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d\": container with ID starting with 652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d not found: ID does not exist" Jul 16 09:11:31.643601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.643558 2567 scope.go:117] "RemoveContainer" containerID="f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e" Jul 16 09:11:31.643762 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.643748 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e"} err="failed to get container status \"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e\": rpc error: code = NotFound desc = could not find container \"f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e\": container with ID starting with f3bf54e50feb6dba961c0cf6ac0e6163f707cc7e69f50ec0041f272c87b9ee4e not found: ID does not exist" Jul 16 09:11:31.643801 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.643762 2567 scope.go:117] "RemoveContainer" containerID="26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d" Jul 16 09:11:31.643995 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.643978 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d"} err="failed to get container status \"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d\": rpc error: code = NotFound desc = could not find container \"26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d\": container with ID starting with 26788aac604f8f45448f0d62874bbafe0806d17e560add8535487c8ef5304f9d not found: ID does not exist" Jul 16 09:11:31.643995 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.643993 2567 scope.go:117] "RemoveContainer" containerID="5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c" Jul 16 09:11:31.644184 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.644168 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c"} err="failed to get container status \"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c\": rpc error: code = NotFound desc = could not find container \"5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c\": container with ID starting with 5386cd7aec4cfb3e57095591504ec5dc10eef68bc3f5ec8cc32bb79f252b5d4c not found: ID does not exist" Jul 16 09:11:31.644184 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.644184 2567 scope.go:117] "RemoveContainer" containerID="450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6" Jul 16 09:11:31.644402 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.644380 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6"} err="failed to get container status \"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6\": rpc error: code = NotFound desc = could not find container \"450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6\": container with ID starting with 450fd509051fefa1d0b75ccf6ea36941b742f759de7c65451250baf842e154a6 not found: ID does not exist" Jul 16 09:11:31.644443 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.644403 2567 scope.go:117] "RemoveContainer" containerID="46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50" Jul 16 09:11:31.644645 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.644628 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50"} err="failed to get container status \"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50\": rpc error: code = NotFound desc = could not find container \"46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50\": container with ID starting with 46e397d8ec61c9f133f6c482239adf9158812132b79fd8b9bfcb8d5428262d50 not found: ID does not exist" Jul 16 09:11:31.644682 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.644646 2567 scope.go:117] "RemoveContainer" containerID="4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30" Jul 16 09:11:31.644867 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.644850 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30"} err="failed to get container status \"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30\": rpc error: code = NotFound desc = could not find container \"4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30\": container with ID starting with 4265adfbf087f0dd5627a739f5f7fccb02430a90322210c1b97cbf910a226a30 not found: ID does not exist" Jul 16 09:11:31.644915 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.644869 2567 scope.go:117] "RemoveContainer" containerID="652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d" Jul 16 09:11:31.645076 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.645059 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d"} err="failed to get container status \"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d\": rpc error: code = NotFound desc = could not find container \"652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d\": container with ID starting with 652193f343810ae824a7c2801413cbd83dcd7e24d32b429e44a1e9afc43da47d not found: ID does not exist" Jul 16 09:11:31.654626 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.654607 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 09:11:31.654932 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.654921 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="kube-rbac-proxy-thanos" Jul 16 09:11:31.654972 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.654934 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="kube-rbac-proxy-thanos" Jul 16 09:11:31.654972 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.654945 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="kube-rbac-proxy-web" Jul 16 09:11:31.654972 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.654951 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="kube-rbac-proxy-web" Jul 16 09:11:31.654972 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.654960 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="config-reloader" Jul 16 09:11:31.654972 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.654966 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="config-reloader" Jul 16 09:11:31.655114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.654974 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="kube-rbac-proxy" Jul 16 09:11:31.655114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.654979 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="kube-rbac-proxy" Jul 16 09:11:31.655114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.654987 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="prometheus" Jul 16 09:11:31.655114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.654992 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="prometheus" Jul 16 09:11:31.655114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.655000 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="init-config-reloader" Jul 16 09:11:31.655114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.655005 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="init-config-reloader" Jul 16 09:11:31.655114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.655012 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="thanos-sidecar" Jul 16 09:11:31.655114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.655016 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="thanos-sidecar" Jul 16 09:11:31.655114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.655066 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="prometheus" Jul 16 09:11:31.655114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.655073 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="kube-rbac-proxy-thanos" Jul 16 09:11:31.655114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.655079 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="config-reloader" Jul 16 09:11:31.655114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.655085 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="kube-rbac-proxy-web" Jul 16 09:11:31.655114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.655092 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="kube-rbac-proxy" Jul 16 09:11:31.655114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.655097 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" containerName="thanos-sidecar" Jul 16 09:11:31.658679 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.658665 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.670807 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.670785 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 16 09:11:31.670807 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.670810 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 16 09:11:31.670987 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.670856 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 16 09:11:31.670987 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.670869 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 16 09:11:31.670987 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.670901 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 16 09:11:31.670987 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.670808 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 16 09:11:31.670987 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.670952 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 16 09:11:31.670987 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.670973 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 16 09:11:31.671199 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.671074 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-53q61v5abv5u\"" Jul 16 09:11:31.671510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.671496 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 16 09:11:31.671607 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.671594 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 16 09:11:31.671676 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.671662 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 16 09:11:31.671729 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.671679 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 16 09:11:31.672115 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.672098 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mlpvb\"" Jul 16 09:11:31.677202 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.677186 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 16 09:11:31.677269 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.677231 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 16 09:11:31.692621 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.692601 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.692730 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.692635 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd854340-3c36-4687-bdbc-aa05194036ee-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.692730 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.692661 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.692852 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.692713 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.692852 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.692763 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.692852 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.692792 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.692943 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.692887 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.692943 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.692915 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.692943 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.692935 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.693046 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.692963 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.693046 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.693019 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.693130 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.693052 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.693130 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.693078 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.693130 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.693100 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wfxs\" (UniqueName: \"kubernetes.io/projected/dd854340-3c36-4687-bdbc-aa05194036ee-kube-api-access-6wfxs\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.693232 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.693141 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.693232 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.693172 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd854340-3c36-4687-bdbc-aa05194036ee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.693232 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.693207 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-config\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.693327 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.693241 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.693327 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.693286 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.693327 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.693311 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.694166 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.694148 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 09:11:31.794042 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.794002 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.794243 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.794049 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd854340-3c36-4687-bdbc-aa05194036ee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.794243 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.794073 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-config\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.794243 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.794100 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.794243 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.794132 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.794243 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.794158 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.794243 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.794191 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.794243 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.794219 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd854340-3c36-4687-bdbc-aa05194036ee-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.794961 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.794932 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.795086 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.795020 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.795086 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.795070 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.795203 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.795107 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.795203 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.795156 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.795203 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.795181 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.795361 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.795228 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.795361 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.795250 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.795361 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.795299 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.795361 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.795326 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.795563 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.795376 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.795563 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.795428 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.795563 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.795473 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.795563 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.795499 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.795563 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.795523 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wfxs\" (UniqueName: \"kubernetes.io/projected/dd854340-3c36-4687-bdbc-aa05194036ee-kube-api-access-6wfxs\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.797214 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.796425 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.797214 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.796971 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.797214 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.797080 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.797214 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.797145 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.797214 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.797161 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd854340-3c36-4687-bdbc-aa05194036ee-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.798160 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.798112 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd854340-3c36-4687-bdbc-aa05194036ee-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.798614 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.798585 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.799276 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.799247 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.799384 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.799308 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-config\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.799384 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.799331 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd854340-3c36-4687-bdbc-aa05194036ee-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.799658 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.799637 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.799717 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.799673 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.799717 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.799705 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.800423 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.800404 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.800800 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.800783 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.800907 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.800803 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd854340-3c36-4687-bdbc-aa05194036ee-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.804798 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.804778 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wfxs\" (UniqueName: \"kubernetes.io/projected/dd854340-3c36-4687-bdbc-aa05194036ee-kube-api-access-6wfxs\") pod \"prometheus-k8s-0\" (UID: \"dd854340-3c36-4687-bdbc-aa05194036ee\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:31.967625 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:31.967525 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:32.108793 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:32.108762 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 09:11:32.111501 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:11:32.111471 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd854340_3c36_4687_bdbc_aa05194036ee.slice/crio-fbd091f49f8c931f901aa2ba38742c9d572d9a9f58cfeab2fdc7b6f74235df33 WatchSource:0}: Error finding container fbd091f49f8c931f901aa2ba38742c9d572d9a9f58cfeab2fdc7b6f74235df33: Status 404 returned error can't find the container with id fbd091f49f8c931f901aa2ba38742c9d572d9a9f58cfeab2fdc7b6f74235df33 Jul 16 09:11:32.582747 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:32.582710 2567 generic.go:356] "Generic (PLEG): container finished" podID="dd854340-3c36-4687-bdbc-aa05194036ee" containerID="d22c1d795a8255c0d087f5a4557e7191d1cbe42ba940feb36d5ecbe66ae25f3d" exitCode=0 Jul 16 09:11:32.583128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:32.582790 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd854340-3c36-4687-bdbc-aa05194036ee","Type":"ContainerDied","Data":"d22c1d795a8255c0d087f5a4557e7191d1cbe42ba940feb36d5ecbe66ae25f3d"} Jul 16 09:11:32.583128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:32.582880 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd854340-3c36-4687-bdbc-aa05194036ee","Type":"ContainerStarted","Data":"fbd091f49f8c931f901aa2ba38742c9d572d9a9f58cfeab2fdc7b6f74235df33"} Jul 16 09:11:33.323140 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:33.323108 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c" path="/var/lib/kubelet/pods/a7b356f7-80d7-4a67-bf0d-fa79cd3a3b8c/volumes" Jul 16 09:11:33.590348 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:33.590271 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd854340-3c36-4687-bdbc-aa05194036ee","Type":"ContainerStarted","Data":"0dc3d47ca0927ea751d1c76ee49e33be4c43f3f8693b8774c451798d40724d86"} Jul 16 09:11:33.590348 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:33.590335 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd854340-3c36-4687-bdbc-aa05194036ee","Type":"ContainerStarted","Data":"9dca596035760a82b76f872686122dda0a90aec71e75e3613a1d3e2299f5e6be"} Jul 16 09:11:33.590727 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:33.590371 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd854340-3c36-4687-bdbc-aa05194036ee","Type":"ContainerStarted","Data":"0390f5e7e0f49f1ccbaa7a823fe9bd210476163daf1a5aaeeed640eb967ed65a"} Jul 16 09:11:33.590727 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:33.590397 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd854340-3c36-4687-bdbc-aa05194036ee","Type":"ContainerStarted","Data":"042d6417a01940dd29b0d465e2410967286fc6ce94b6423c5f07fe158d64d307"} Jul 16 09:11:33.590727 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:33.590419 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd854340-3c36-4687-bdbc-aa05194036ee","Type":"ContainerStarted","Data":"d3f0df89cd55d27e97cd6b2420ea5a0d866324a8b0cdf688c620a3959cfae66c"} Jul 16 09:11:33.590727 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:33.590451 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd854340-3c36-4687-bdbc-aa05194036ee","Type":"ContainerStarted","Data":"c4cd57c8b41e9f5b5eb65aa1c412bc5780f8dacfb89871fc3a8d82f92c0ec074"} Jul 16 09:11:33.638172 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:33.638121 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.619333545 podStartE2EDuration="2.638104328s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.770783ms" isStatefulPod=false podCreationTimestamp="2026-07-16 09:11:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:11:33.637075743 +0000 UTC m=+369.046624003" watchObservedRunningTime="2026-07-16 09:11:33.638104328 +0000 UTC m=+369.047652587" Jul 16 09:11:36.967835 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:36.967766 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:11:38.050364 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:38.050332 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:11:38.052686 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:38.052649 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd-metrics-certs\") pod \"network-metrics-daemon-ph7bq\" (UID: \"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd\") " pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:11:38.226583 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:38.226551 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h777f\"" Jul 16 09:11:38.234769 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:38.234748 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ph7bq" Jul 16 09:11:38.360852 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:38.360746 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ph7bq"] Jul 16 09:11:38.363585 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:11:38.363556 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f9f96c9_54b3_47a9_9329_6ac7e8ded7bd.slice/crio-432d2f99a15bfafd370b7d379609c36eda5f82516b3ef350f6e91413e44d54e5 WatchSource:0}: Error finding container 432d2f99a15bfafd370b7d379609c36eda5f82516b3ef350f6e91413e44d54e5: Status 404 returned error can't find the container with id 432d2f99a15bfafd370b7d379609c36eda5f82516b3ef350f6e91413e44d54e5 Jul 16 09:11:38.365445 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:38.365430 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 09:11:38.614011 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:38.613919 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ph7bq" event={"ID":"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd","Type":"ContainerStarted","Data":"432d2f99a15bfafd370b7d379609c36eda5f82516b3ef350f6e91413e44d54e5"} Jul 16 09:11:40.624606 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:40.624514 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ph7bq" event={"ID":"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd","Type":"ContainerStarted","Data":"0c859fdb6a86dcbaf6bf550f9428aa0243c67a417581d66eefeaa2267a094a27"} Jul 16 09:11:40.624606 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:40.624576 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ph7bq" event={"ID":"9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd","Type":"ContainerStarted","Data":"a60e03e4fe237fefc74067259b295a1ab5a002301b08e01338d5fae4deff5be1"} Jul 16 09:11:40.642867 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:11:40.642806 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ph7bq" podStartSLOduration=373.649232445 podStartE2EDuration="6m15.642790353s" totalImagesPullingTime="1.993557908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:11:40.642328654 +0000 UTC m=+376.051876910" watchObservedRunningTime="2026-07-16 09:11:40.642790353 +0000 UTC m=+376.052338613" Jul 16 09:12:31.967740 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:12:31.967701 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:12:31.983436 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:12:31.983411 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:12:32.850126 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:12:32.850095 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 09:15:04.474836 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.474783 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf"] Jul 16 09:15:04.482470 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.482440 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.486122 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.486093 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 09:15:04.486441 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.486405 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 09:15:04.486584 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.486564 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 09:15:04.486706 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.486539 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-m6hkx\"" Jul 16 09:15:04.486706 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.486490 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 09:15:04.494498 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.494475 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf"] Jul 16 09:15:04.539410 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.539374 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6f0706de-4ebf-4623-b150-50b95a7bc8ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.539410 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.539413 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f0706de-4ebf-4623-b150-50b95a7bc8ef-tmp\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.539655 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.539435 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m4gz\" (UniqueName: \"kubernetes.io/projected/6f0706de-4ebf-4623-b150-50b95a7bc8ef-kube-api-access-4m4gz\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.539655 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.539540 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f0706de-4ebf-4623-b150-50b95a7bc8ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.539655 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.539589 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6f0706de-4ebf-4623-b150-50b95a7bc8ef-manifests\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.640964 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.640927 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6f0706de-4ebf-4623-b150-50b95a7bc8ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.640964 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.640968 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f0706de-4ebf-4623-b150-50b95a7bc8ef-tmp\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.641241 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.640994 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m4gz\" (UniqueName: \"kubernetes.io/projected/6f0706de-4ebf-4623-b150-50b95a7bc8ef-kube-api-access-4m4gz\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.641241 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.641028 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f0706de-4ebf-4623-b150-50b95a7bc8ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.641241 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.641061 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6f0706de-4ebf-4623-b150-50b95a7bc8ef-manifests\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.641509 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.641486 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6f0706de-4ebf-4623-b150-50b95a7bc8ef-manifests\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.643435 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.643406 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f0706de-4ebf-4623-b150-50b95a7bc8ef-tmp\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.643532 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.643497 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6f0706de-4ebf-4623-b150-50b95a7bc8ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.643571 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.643543 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f0706de-4ebf-4623-b150-50b95a7bc8ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.649166 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.649139 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m4gz\" (UniqueName: \"kubernetes.io/projected/6f0706de-4ebf-4623-b150-50b95a7bc8ef-kube-api-access-4m4gz\") pod \"opendatahub-operator-controller-manager-6d64bd87c6-jdlvf\" (UID: \"6f0706de-4ebf-4623-b150-50b95a7bc8ef\") " pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.794504 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.794468 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:04.935696 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:04.935669 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf"] Jul 16 09:15:04.938412 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:15:04.938381 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f0706de_4ebf_4623_b150_50b95a7bc8ef.slice/crio-9a92e5bc49a17457dc36f9c240e12f6c3a1189d388a28c15b5293d81d480b6aa WatchSource:0}: Error finding container 9a92e5bc49a17457dc36f9c240e12f6c3a1189d388a28c15b5293d81d480b6aa: Status 404 returned error can't find the container with id 9a92e5bc49a17457dc36f9c240e12f6c3a1189d388a28c15b5293d81d480b6aa Jul 16 09:15:05.472161 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:05.472120 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" event={"ID":"6f0706de-4ebf-4623-b150-50b95a7bc8ef","Type":"ContainerStarted","Data":"9a92e5bc49a17457dc36f9c240e12f6c3a1189d388a28c15b5293d81d480b6aa"} Jul 16 09:15:10.501009 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:10.500966 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" event={"ID":"6f0706de-4ebf-4623-b150-50b95a7bc8ef","Type":"ContainerStarted","Data":"348a790b082006d998cef6bbbcd80a6a2f3ad2a29aba1afc9e2a5dd6632a9f44"} Jul 16 09:15:11.506726 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:11.506690 2567 generic.go:356] "Generic (PLEG): container finished" podID="6f0706de-4ebf-4623-b150-50b95a7bc8ef" containerID="348a790b082006d998cef6bbbcd80a6a2f3ad2a29aba1afc9e2a5dd6632a9f44" exitCode=0 Jul 16 09:15:11.507151 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:11.506780 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" event={"ID":"6f0706de-4ebf-4623-b150-50b95a7bc8ef","Type":"ContainerDied","Data":"348a790b082006d998cef6bbbcd80a6a2f3ad2a29aba1afc9e2a5dd6632a9f44"} Jul 16 09:15:12.513982 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:12.513945 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" event={"ID":"6f0706de-4ebf-4623-b150-50b95a7bc8ef","Type":"ContainerStarted","Data":"6d286e0833e2526f1fbf2945a3f4f9922bad95bef14601eab7e123e217f94979"} Jul 16 09:15:12.514372 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:12.514061 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:12.536862 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:12.536799 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" podStartSLOduration=2.449734673 podStartE2EDuration="8.536781712s" totalImagesPullingTime="5.029587623s" totalInitContainerRuntime="1.057459416s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:15:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:15:12.535637035 +0000 UTC m=+587.945185308" watchObservedRunningTime="2026-07-16 09:15:12.536781712 +0000 UTC m=+587.946329972" Jul 16 09:15:23.521485 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:23.521457 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d64bd87c6-jdlvf" Jul 16 09:15:25.111682 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:25.111652 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:15:25.112930 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:25.112899 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:15:27.366438 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:27.366394 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9tw24"] Jul 16 09:15:27.372809 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:27.372781 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9tw24" Jul 16 09:15:27.376703 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:27.376681 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 16 09:15:27.376703 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:27.376691 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-rg9gm\"" Jul 16 09:15:27.383990 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:27.383961 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9tw24"] Jul 16 09:15:27.551989 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:27.551951 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a52779e-3df9-43cc-b885-6b04b6b1e9f0-tls-certs\") pod \"model-serving-api-54d89bd5fc-9tw24\" (UID: \"9a52779e-3df9-43cc-b885-6b04b6b1e9f0\") " pod="opendatahub/model-serving-api-54d89bd5fc-9tw24" Jul 16 09:15:27.552198 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:27.552018 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc7s9\" (UniqueName: \"kubernetes.io/projected/9a52779e-3df9-43cc-b885-6b04b6b1e9f0-kube-api-access-fc7s9\") pod \"model-serving-api-54d89bd5fc-9tw24\" (UID: \"9a52779e-3df9-43cc-b885-6b04b6b1e9f0\") " pod="opendatahub/model-serving-api-54d89bd5fc-9tw24" Jul 16 09:15:27.653090 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:27.652979 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc7s9\" (UniqueName: \"kubernetes.io/projected/9a52779e-3df9-43cc-b885-6b04b6b1e9f0-kube-api-access-fc7s9\") pod \"model-serving-api-54d89bd5fc-9tw24\" (UID: \"9a52779e-3df9-43cc-b885-6b04b6b1e9f0\") " pod="opendatahub/model-serving-api-54d89bd5fc-9tw24" Jul 16 09:15:27.653275 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:27.653098 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a52779e-3df9-43cc-b885-6b04b6b1e9f0-tls-certs\") pod \"model-serving-api-54d89bd5fc-9tw24\" (UID: \"9a52779e-3df9-43cc-b885-6b04b6b1e9f0\") " pod="opendatahub/model-serving-api-54d89bd5fc-9tw24" Jul 16 09:15:27.653275 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:15:27.653243 2567 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 16 09:15:27.653387 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:15:27.653307 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a52779e-3df9-43cc-b885-6b04b6b1e9f0-tls-certs podName:9a52779e-3df9-43cc-b885-6b04b6b1e9f0 nodeName:}" failed. No retries permitted until 2026-07-16 09:15:28.153286076 +0000 UTC m=+603.562834317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9a52779e-3df9-43cc-b885-6b04b6b1e9f0-tls-certs") pod "model-serving-api-54d89bd5fc-9tw24" (UID: "9a52779e-3df9-43cc-b885-6b04b6b1e9f0") : secret "model-serving-api-tls" not found Jul 16 09:15:27.664996 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:27.664961 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc7s9\" (UniqueName: \"kubernetes.io/projected/9a52779e-3df9-43cc-b885-6b04b6b1e9f0-kube-api-access-fc7s9\") pod \"model-serving-api-54d89bd5fc-9tw24\" (UID: \"9a52779e-3df9-43cc-b885-6b04b6b1e9f0\") " pod="opendatahub/model-serving-api-54d89bd5fc-9tw24" Jul 16 09:15:28.157923 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:28.157859 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a52779e-3df9-43cc-b885-6b04b6b1e9f0-tls-certs\") pod \"model-serving-api-54d89bd5fc-9tw24\" (UID: \"9a52779e-3df9-43cc-b885-6b04b6b1e9f0\") " pod="opendatahub/model-serving-api-54d89bd5fc-9tw24" Jul 16 09:15:28.160284 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:28.160255 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a52779e-3df9-43cc-b885-6b04b6b1e9f0-tls-certs\") pod \"model-serving-api-54d89bd5fc-9tw24\" (UID: \"9a52779e-3df9-43cc-b885-6b04b6b1e9f0\") " pod="opendatahub/model-serving-api-54d89bd5fc-9tw24" Jul 16 09:15:28.283804 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:28.283770 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9tw24" Jul 16 09:15:28.416892 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:28.416863 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9tw24"] Jul 16 09:15:28.418919 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:15:28.418897 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a52779e_3df9_43cc_b885_6b04b6b1e9f0.slice/crio-b2cd822f7d155018e9f345624ca168ef15ac9c4b072c99cf01965b0cb9fdfc71 WatchSource:0}: Error finding container b2cd822f7d155018e9f345624ca168ef15ac9c4b072c99cf01965b0cb9fdfc71: Status 404 returned error can't find the container with id b2cd822f7d155018e9f345624ca168ef15ac9c4b072c99cf01965b0cb9fdfc71 Jul 16 09:15:28.592289 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:28.592250 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9tw24" event={"ID":"9a52779e-3df9-43cc-b885-6b04b6b1e9f0","Type":"ContainerStarted","Data":"b2cd822f7d155018e9f345624ca168ef15ac9c4b072c99cf01965b0cb9fdfc71"} Jul 16 09:15:31.607780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:31.607692 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9tw24" event={"ID":"9a52779e-3df9-43cc-b885-6b04b6b1e9f0","Type":"ContainerStarted","Data":"931309a9b031719093690d908bcde0daf9294755b93f9b70fb6dd64bc10bd5ab"} Jul 16 09:15:31.607780 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:31.607756 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9tw24" Jul 16 09:15:31.627586 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:31.627525 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9tw24" podStartSLOduration=1.7226827390000001 podStartE2EDuration="4.627508712s" totalImagesPullingTime="2.904825973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:15:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:15:31.626234962 +0000 UTC m=+607.035783237" watchObservedRunningTime="2026-07-16 09:15:31.627508712 +0000 UTC m=+607.037056971" Jul 16 09:15:34.219343 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.219306 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb"] Jul 16 09:15:34.222852 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.222810 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:34.226306 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.226285 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 16 09:15:34.226858 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.226806 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 09:15:34.227531 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.227513 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tnmk8\"" Jul 16 09:15:34.228003 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.227983 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 16 09:15:34.228097 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.227983 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 09:15:34.228097 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.228012 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 16 09:15:34.233571 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.233548 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb"] Jul 16 09:15:34.312143 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.312108 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59s68\" (UniqueName: \"kubernetes.io/projected/7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40-kube-api-access-59s68\") pod \"lws-controller-manager-6ddb4bcd65-ktfzb\" (UID: \"7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40\") " pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:34.312324 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.312162 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40-metrics-cert\") pod \"lws-controller-manager-6ddb4bcd65-ktfzb\" (UID: \"7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40\") " pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:34.312324 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.312247 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40-cert\") pod \"lws-controller-manager-6ddb4bcd65-ktfzb\" (UID: \"7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40\") " pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:34.312324 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.312296 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40-manager-config\") pod \"lws-controller-manager-6ddb4bcd65-ktfzb\" (UID: \"7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40\") " pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:34.413396 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.413358 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40-manager-config\") pod \"lws-controller-manager-6ddb4bcd65-ktfzb\" (UID: \"7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40\") " pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:34.413584 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.413416 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59s68\" (UniqueName: \"kubernetes.io/projected/7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40-kube-api-access-59s68\") pod \"lws-controller-manager-6ddb4bcd65-ktfzb\" (UID: \"7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40\") " pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:34.413584 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.413454 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40-metrics-cert\") pod \"lws-controller-manager-6ddb4bcd65-ktfzb\" (UID: \"7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40\") " pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:34.413584 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.413494 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40-cert\") pod \"lws-controller-manager-6ddb4bcd65-ktfzb\" (UID: \"7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40\") " pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:34.414124 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.414098 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40-manager-config\") pod \"lws-controller-manager-6ddb4bcd65-ktfzb\" (UID: \"7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40\") " pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:34.416007 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.415976 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40-cert\") pod \"lws-controller-manager-6ddb4bcd65-ktfzb\" (UID: \"7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40\") " pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:34.416007 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.415999 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40-metrics-cert\") pod \"lws-controller-manager-6ddb4bcd65-ktfzb\" (UID: \"7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40\") " pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:34.446143 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.446114 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59s68\" (UniqueName: \"kubernetes.io/projected/7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40-kube-api-access-59s68\") pod \"lws-controller-manager-6ddb4bcd65-ktfzb\" (UID: \"7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40\") " pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:34.533586 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.533548 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:34.671692 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:34.671670 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb"] Jul 16 09:15:34.674084 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:15:34.674052 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dc06d9a_8a07_4ca6_b2b4_b7b133e68c40.slice/crio-a346906971a477b16259eef276bf09207863178eff6db5e5182fd607c2776b90 WatchSource:0}: Error finding container a346906971a477b16259eef276bf09207863178eff6db5e5182fd607c2776b90: Status 404 returned error can't find the container with id a346906971a477b16259eef276bf09207863178eff6db5e5182fd607c2776b90 Jul 16 09:15:35.259678 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.259638 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df"] Jul 16 09:15:35.272042 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.267099 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.275805 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.273707 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 09:15:35.277976 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.276932 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8frkp\"" Jul 16 09:15:35.277976 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.277128 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 09:15:35.277976 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.277300 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 09:15:35.277976 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.277492 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 09:15:35.277976 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.277677 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 09:15:35.283364 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.282684 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 09:15:35.285265 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.284803 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df"] Jul 16 09:15:35.321303 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.321079 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8e13b25-89a1-4473-9566-1d0f0aeab39b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.321303 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.321121 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c8e13b25-89a1-4473-9566-1d0f0aeab39b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.321303 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.321154 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c8e13b25-89a1-4473-9566-1d0f0aeab39b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.321303 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.321187 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c8e13b25-89a1-4473-9566-1d0f0aeab39b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.321303 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.321211 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb9qs\" (UniqueName: \"kubernetes.io/projected/c8e13b25-89a1-4473-9566-1d0f0aeab39b-kube-api-access-sb9qs\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.321728 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.321344 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8e13b25-89a1-4473-9566-1d0f0aeab39b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.321728 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.321398 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c8e13b25-89a1-4473-9566-1d0f0aeab39b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.422092 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.422041 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8e13b25-89a1-4473-9566-1d0f0aeab39b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.422092 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.422093 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c8e13b25-89a1-4473-9566-1d0f0aeab39b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.422339 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.422121 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c8e13b25-89a1-4473-9566-1d0f0aeab39b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.422339 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.422161 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c8e13b25-89a1-4473-9566-1d0f0aeab39b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.422339 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.422187 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb9qs\" (UniqueName: \"kubernetes.io/projected/c8e13b25-89a1-4473-9566-1d0f0aeab39b-kube-api-access-sb9qs\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.422339 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.422271 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8e13b25-89a1-4473-9566-1d0f0aeab39b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.422339 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.422324 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c8e13b25-89a1-4473-9566-1d0f0aeab39b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.423413 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.423354 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c8e13b25-89a1-4473-9566-1d0f0aeab39b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.425735 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.425701 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c8e13b25-89a1-4473-9566-1d0f0aeab39b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.426716 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.426681 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8e13b25-89a1-4473-9566-1d0f0aeab39b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.428475 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.428451 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c8e13b25-89a1-4473-9566-1d0f0aeab39b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.431013 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.430987 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c8e13b25-89a1-4473-9566-1d0f0aeab39b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.435913 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.435888 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8e13b25-89a1-4473-9566-1d0f0aeab39b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.439163 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.439110 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb9qs\" (UniqueName: \"kubernetes.io/projected/c8e13b25-89a1-4473-9566-1d0f0aeab39b-kube-api-access-sb9qs\") pod \"istiod-openshift-gateway-94bb8fbfd-xd8df\" (UID: \"c8e13b25-89a1-4473-9566-1d0f0aeab39b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.593366 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.593330 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:35.630569 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.630534 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" event={"ID":"7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40","Type":"ContainerStarted","Data":"a346906971a477b16259eef276bf09207863178eff6db5e5182fd607c2776b90"} Jul 16 09:15:35.748582 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:35.748545 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df"] Jul 16 09:15:36.675786 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:15:36.675751 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e13b25_89a1_4473_9566_1d0f0aeab39b.slice/crio-6f124a8a5dc41f6064afa9cb38290a027d1dcc42cc00df7e94f2c7bad707d8e5 WatchSource:0}: Error finding container 6f124a8a5dc41f6064afa9cb38290a027d1dcc42cc00df7e94f2c7bad707d8e5: Status 404 returned error can't find the container with id 6f124a8a5dc41f6064afa9cb38290a027d1dcc42cc00df7e94f2c7bad707d8e5 Jul 16 09:15:37.644266 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:37.644226 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" event={"ID":"7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40","Type":"ContainerStarted","Data":"24d99b8a90aac57b47cc8ddcc30e5ec2be1c5c39b6171b1eed5fa47e335317b3"} Jul 16 09:15:37.644456 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:37.644292 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:15:37.646311 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:37.646268 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" event={"ID":"c8e13b25-89a1-4473-9566-1d0f0aeab39b","Type":"ContainerStarted","Data":"6f124a8a5dc41f6064afa9cb38290a027d1dcc42cc00df7e94f2c7bad707d8e5"} Jul 16 09:15:37.689524 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:37.689462 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" podStartSLOduration=1.620135914 podStartE2EDuration="3.689439298s" totalImagesPullingTime="2.069303384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:15:37.688227795 +0000 UTC m=+613.097776259" watchObservedRunningTime="2026-07-16 09:15:37.689439298 +0000 UTC m=+613.098987560" Jul 16 09:15:39.019738 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:39.019694 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 09:15:39.020067 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:39.019766 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 09:15:39.659150 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:39.659118 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" event={"ID":"c8e13b25-89a1-4473-9566-1d0f0aeab39b","Type":"ContainerStarted","Data":"6bb943dd0c514b141c4093ed24ed4c671857a2e6f017ddfa6ead5032e957fd76"} Jul 16 09:15:39.659319 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:39.659210 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:39.685024 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:39.684961 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" podStartSLOduration=2.343722745 podStartE2EDuration="4.684941797s" totalImagesPullingTime="2.341219052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:15:39.684050027 +0000 UTC m=+615.093598312" watchObservedRunningTime="2026-07-16 09:15:39.684941797 +0000 UTC m=+615.094490058" Jul 16 09:15:40.668682 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:40.668652 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xd8df" Jul 16 09:15:42.616859 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:42.616804 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9tw24" Jul 16 09:15:48.653886 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:15:48.653765 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6ddb4bcd65-ktfzb" Jul 16 09:17:20.470177 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.470132 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng"] Jul 16 09:17:20.472854 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.472805 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" Jul 16 09:17:20.476715 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.476688 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 09:17:20.476923 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.476904 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z6zqn\"" Jul 16 09:17:20.477004 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.476953 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 09:17:20.490131 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.490102 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng"] Jul 16 09:17:20.593561 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.593521 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91dfd102-3f92-4dc9-badf-bb42bd6d5595-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vdkng\" (UID: \"91dfd102-3f92-4dc9-badf-bb42bd6d5595\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" Jul 16 09:17:20.593738 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.593588 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj5j8\" (UniqueName: \"kubernetes.io/projected/91dfd102-3f92-4dc9-badf-bb42bd6d5595-kube-api-access-jj5j8\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vdkng\" (UID: \"91dfd102-3f92-4dc9-badf-bb42bd6d5595\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" Jul 16 09:17:20.694554 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.694512 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj5j8\" (UniqueName: \"kubernetes.io/projected/91dfd102-3f92-4dc9-badf-bb42bd6d5595-kube-api-access-jj5j8\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vdkng\" (UID: \"91dfd102-3f92-4dc9-badf-bb42bd6d5595\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" Jul 16 09:17:20.694735 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.694644 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91dfd102-3f92-4dc9-badf-bb42bd6d5595-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vdkng\" (UID: \"91dfd102-3f92-4dc9-badf-bb42bd6d5595\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" Jul 16 09:17:20.695063 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.695045 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91dfd102-3f92-4dc9-badf-bb42bd6d5595-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vdkng\" (UID: \"91dfd102-3f92-4dc9-badf-bb42bd6d5595\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" Jul 16 09:17:20.704960 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.704936 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj5j8\" (UniqueName: \"kubernetes.io/projected/91dfd102-3f92-4dc9-badf-bb42bd6d5595-kube-api-access-jj5j8\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vdkng\" (UID: \"91dfd102-3f92-4dc9-badf-bb42bd6d5595\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" Jul 16 09:17:20.784189 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.784141 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" Jul 16 09:17:20.920503 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.920471 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng"] Jul 16 09:17:20.922566 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:17:20.922541 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91dfd102_3f92_4dc9_badf_bb42bd6d5595.slice/crio-e4496e70a41b1f0c2bca21acdadcfc6792aa55733d4fdad13dab410b42b8dab6 WatchSource:0}: Error finding container e4496e70a41b1f0c2bca21acdadcfc6792aa55733d4fdad13dab410b42b8dab6: Status 404 returned error can't find the container with id e4496e70a41b1f0c2bca21acdadcfc6792aa55733d4fdad13dab410b42b8dab6 Jul 16 09:17:20.925607 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:20.925588 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 09:17:21.143974 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:21.143886 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" event={"ID":"91dfd102-3f92-4dc9-badf-bb42bd6d5595","Type":"ContainerStarted","Data":"e4496e70a41b1f0c2bca21acdadcfc6792aa55733d4fdad13dab410b42b8dab6"} Jul 16 09:17:26.172226 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:26.172188 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" event={"ID":"91dfd102-3f92-4dc9-badf-bb42bd6d5595","Type":"ContainerStarted","Data":"f6bd05b0f7f5e8377b71426dd35e9cca0be7aee8b8b894a4df8cda7adc52cbfc"} Jul 16 09:17:26.172696 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:26.172301 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" Jul 16 09:17:26.197177 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:26.197128 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" podStartSLOduration=1.777458937 podStartE2EDuration="6.197111205s" totalImagesPullingTime="4.419652268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:17:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:17:26.194289573 +0000 UTC m=+721.603837834" watchObservedRunningTime="2026-07-16 09:17:26.197111205 +0000 UTC m=+721.606659465" Jul 16 09:17:37.178707 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:37.178676 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" Jul 16 09:17:49.936840 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:49.936675 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng"] Jul 16 09:17:49.940840 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:49.937272 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" podUID="91dfd102-3f92-4dc9-badf-bb42bd6d5595" containerName="manager" containerID="cri-o://f6bd05b0f7f5e8377b71426dd35e9cca0be7aee8b8b894a4df8cda7adc52cbfc" gracePeriod=10 Jul 16 09:17:50.199500 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.199436 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" Jul 16 09:17:50.262893 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.262857 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj5j8\" (UniqueName: \"kubernetes.io/projected/91dfd102-3f92-4dc9-badf-bb42bd6d5595-kube-api-access-jj5j8\") pod \"91dfd102-3f92-4dc9-badf-bb42bd6d5595\" (UID: \"91dfd102-3f92-4dc9-badf-bb42bd6d5595\") " Jul 16 09:17:50.263084 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.262961 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91dfd102-3f92-4dc9-badf-bb42bd6d5595-extensions-socket-volume\") pod \"91dfd102-3f92-4dc9-badf-bb42bd6d5595\" (UID: \"91dfd102-3f92-4dc9-badf-bb42bd6d5595\") " Jul 16 09:17:50.263449 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.263421 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91dfd102-3f92-4dc9-badf-bb42bd6d5595-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "91dfd102-3f92-4dc9-badf-bb42bd6d5595" (UID: "91dfd102-3f92-4dc9-badf-bb42bd6d5595"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 09:17:50.265148 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.265125 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91dfd102-3f92-4dc9-badf-bb42bd6d5595-kube-api-access-jj5j8" (OuterVolumeSpecName: "kube-api-access-jj5j8") pod "91dfd102-3f92-4dc9-badf-bb42bd6d5595" (UID: "91dfd102-3f92-4dc9-badf-bb42bd6d5595"). InnerVolumeSpecName "kube-api-access-jj5j8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 09:17:50.286416 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.286384 2567 generic.go:356] "Generic (PLEG): container finished" podID="91dfd102-3f92-4dc9-badf-bb42bd6d5595" containerID="f6bd05b0f7f5e8377b71426dd35e9cca0be7aee8b8b894a4df8cda7adc52cbfc" exitCode=0 Jul 16 09:17:50.286555 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.286428 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" Jul 16 09:17:50.286555 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.286446 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" event={"ID":"91dfd102-3f92-4dc9-badf-bb42bd6d5595","Type":"ContainerDied","Data":"f6bd05b0f7f5e8377b71426dd35e9cca0be7aee8b8b894a4df8cda7adc52cbfc"} Jul 16 09:17:50.286555 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.286497 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng" event={"ID":"91dfd102-3f92-4dc9-badf-bb42bd6d5595","Type":"ContainerDied","Data":"e4496e70a41b1f0c2bca21acdadcfc6792aa55733d4fdad13dab410b42b8dab6"} Jul 16 09:17:50.286555 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.286521 2567 scope.go:117] "RemoveContainer" containerID="f6bd05b0f7f5e8377b71426dd35e9cca0be7aee8b8b894a4df8cda7adc52cbfc" Jul 16 09:17:50.296363 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.296340 2567 scope.go:117] "RemoveContainer" containerID="f6bd05b0f7f5e8377b71426dd35e9cca0be7aee8b8b894a4df8cda7adc52cbfc" Jul 16 09:17:50.296702 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:17:50.296682 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6bd05b0f7f5e8377b71426dd35e9cca0be7aee8b8b894a4df8cda7adc52cbfc\": container with ID starting with f6bd05b0f7f5e8377b71426dd35e9cca0be7aee8b8b894a4df8cda7adc52cbfc not found: ID does not exist" containerID="f6bd05b0f7f5e8377b71426dd35e9cca0be7aee8b8b894a4df8cda7adc52cbfc" Jul 16 09:17:50.296755 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.296712 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6bd05b0f7f5e8377b71426dd35e9cca0be7aee8b8b894a4df8cda7adc52cbfc"} err="failed to get container status \"f6bd05b0f7f5e8377b71426dd35e9cca0be7aee8b8b894a4df8cda7adc52cbfc\": rpc error: code = NotFound desc = could not find container \"f6bd05b0f7f5e8377b71426dd35e9cca0be7aee8b8b894a4df8cda7adc52cbfc\": container with ID starting with f6bd05b0f7f5e8377b71426dd35e9cca0be7aee8b8b894a4df8cda7adc52cbfc not found: ID does not exist" Jul 16 09:17:50.316325 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.316285 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng"] Jul 16 09:17:50.327363 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.327334 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vdkng"] Jul 16 09:17:50.364259 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.364193 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj5j8\" (UniqueName: \"kubernetes.io/projected/91dfd102-3f92-4dc9-badf-bb42bd6d5595-kube-api-access-jj5j8\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:17:50.364259 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:50.364228 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91dfd102-3f92-4dc9-badf-bb42bd6d5595-extensions-socket-volume\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:17:51.324521 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:17:51.324483 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91dfd102-3f92-4dc9-badf-bb42bd6d5595" path="/var/lib/kubelet/pods/91dfd102-3f92-4dc9-badf-bb42bd6d5595/volumes" Jul 16 09:18:06.505454 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.505412 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fvwz"] Jul 16 09:18:06.506544 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.506513 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91dfd102-3f92-4dc9-badf-bb42bd6d5595" containerName="manager" Jul 16 09:18:06.506544 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.506544 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="91dfd102-3f92-4dc9-badf-bb42bd6d5595" containerName="manager" Jul 16 09:18:06.506806 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.506788 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="91dfd102-3f92-4dc9-badf-bb42bd6d5595" containerName="manager" Jul 16 09:18:06.512425 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.512386 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" Jul 16 09:18:06.515568 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.515537 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 09:18:06.515704 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.515578 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 09:18:06.516558 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.516538 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dnnrv\"" Jul 16 09:18:06.516654 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.516600 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 16 09:18:06.520928 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.520906 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fvwz"] Jul 16 09:18:06.604833 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.604787 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fvwz"] Jul 16 09:18:06.616198 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.616167 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7jhd\" (UniqueName: \"kubernetes.io/projected/f7c3930a-1578-463b-b603-127cf7974b0e-kube-api-access-x7jhd\") pod \"limitador-limitador-64c94fd64f-6fvwz\" (UID: \"f7c3930a-1578-463b-b603-127cf7974b0e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" Jul 16 09:18:06.616353 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.616205 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f7c3930a-1578-463b-b603-127cf7974b0e-config-file\") pod \"limitador-limitador-64c94fd64f-6fvwz\" (UID: \"f7c3930a-1578-463b-b603-127cf7974b0e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" Jul 16 09:18:06.717700 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.717664 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7jhd\" (UniqueName: \"kubernetes.io/projected/f7c3930a-1578-463b-b603-127cf7974b0e-kube-api-access-x7jhd\") pod \"limitador-limitador-64c94fd64f-6fvwz\" (UID: \"f7c3930a-1578-463b-b603-127cf7974b0e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" Jul 16 09:18:06.717700 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.717705 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f7c3930a-1578-463b-b603-127cf7974b0e-config-file\") pod \"limitador-limitador-64c94fd64f-6fvwz\" (UID: \"f7c3930a-1578-463b-b603-127cf7974b0e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" Jul 16 09:18:06.718372 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.718355 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f7c3930a-1578-463b-b603-127cf7974b0e-config-file\") pod \"limitador-limitador-64c94fd64f-6fvwz\" (UID: \"f7c3930a-1578-463b-b603-127cf7974b0e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" Jul 16 09:18:06.726287 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.726260 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7jhd\" (UniqueName: \"kubernetes.io/projected/f7c3930a-1578-463b-b603-127cf7974b0e-kube-api-access-x7jhd\") pod \"limitador-limitador-64c94fd64f-6fvwz\" (UID: \"f7c3930a-1578-463b-b603-127cf7974b0e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" Jul 16 09:18:06.824467 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.824422 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" Jul 16 09:18:06.962284 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:06.962260 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fvwz"] Jul 16 09:18:06.964510 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:18:06.964481 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7c3930a_1578_463b_b603_127cf7974b0e.slice/crio-63315bc8bccc92f22728802456fd44effbfca3c66169e45a857250d5da513627 WatchSource:0}: Error finding container 63315bc8bccc92f22728802456fd44effbfca3c66169e45a857250d5da513627: Status 404 returned error can't find the container with id 63315bc8bccc92f22728802456fd44effbfca3c66169e45a857250d5da513627 Jul 16 09:18:07.367377 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:07.367336 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" event={"ID":"f7c3930a-1578-463b-b603-127cf7974b0e","Type":"ContainerStarted","Data":"63315bc8bccc92f22728802456fd44effbfca3c66169e45a857250d5da513627"} Jul 16 09:18:10.384562 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:10.384516 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" event={"ID":"f7c3930a-1578-463b-b603-127cf7974b0e","Type":"ContainerStarted","Data":"3d0b8a5ec699b9fe6746b5877988ad3a9aa2e3e3bf5b424416170e4119336467"} Jul 16 09:18:10.384947 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:10.384583 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" Jul 16 09:18:10.402338 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:10.402281 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" podStartSLOduration=1.5244784519999999 podStartE2EDuration="4.402264883s" totalImagesPullingTime="2.877786431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:18:10.401406449 +0000 UTC m=+765.810954733" watchObservedRunningTime="2026-07-16 09:18:10.402264883 +0000 UTC m=+765.811813142" Jul 16 09:18:21.390415 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:21.390376 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" Jul 16 09:18:23.129238 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:23.129199 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fvwz"] Jul 16 09:18:23.129664 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:23.129415 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" podUID="f7c3930a-1578-463b-b603-127cf7974b0e" containerName="limitador" containerID="cri-o://3d0b8a5ec699b9fe6746b5877988ad3a9aa2e3e3bf5b424416170e4119336467" gracePeriod=30 Jul 16 09:18:23.683048 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:23.683024 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" Jul 16 09:18:23.686066 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:23.686048 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7jhd\" (UniqueName: \"kubernetes.io/projected/f7c3930a-1578-463b-b603-127cf7974b0e-kube-api-access-x7jhd\") pod \"f7c3930a-1578-463b-b603-127cf7974b0e\" (UID: \"f7c3930a-1578-463b-b603-127cf7974b0e\") " Jul 16 09:18:23.686159 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:23.686107 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f7c3930a-1578-463b-b603-127cf7974b0e-config-file\") pod \"f7c3930a-1578-463b-b603-127cf7974b0e\" (UID: \"f7c3930a-1578-463b-b603-127cf7974b0e\") " Jul 16 09:18:23.686488 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:23.686467 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7c3930a-1578-463b-b603-127cf7974b0e-config-file" (OuterVolumeSpecName: "config-file") pod "f7c3930a-1578-463b-b603-127cf7974b0e" (UID: "f7c3930a-1578-463b-b603-127cf7974b0e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 09:18:23.688257 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:23.688193 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7c3930a-1578-463b-b603-127cf7974b0e-kube-api-access-x7jhd" (OuterVolumeSpecName: "kube-api-access-x7jhd") pod "f7c3930a-1578-463b-b603-127cf7974b0e" (UID: "f7c3930a-1578-463b-b603-127cf7974b0e"). InnerVolumeSpecName "kube-api-access-x7jhd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 09:18:23.787187 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:23.787152 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7jhd\" (UniqueName: \"kubernetes.io/projected/f7c3930a-1578-463b-b603-127cf7974b0e-kube-api-access-x7jhd\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:18:23.787187 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:23.787185 2567 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f7c3930a-1578-463b-b603-127cf7974b0e-config-file\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:18:24.449939 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:24.449905 2567 generic.go:356] "Generic (PLEG): container finished" podID="f7c3930a-1578-463b-b603-127cf7974b0e" containerID="3d0b8a5ec699b9fe6746b5877988ad3a9aa2e3e3bf5b424416170e4119336467" exitCode=0 Jul 16 09:18:24.450352 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:24.449949 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" Jul 16 09:18:24.450352 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:24.449991 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" event={"ID":"f7c3930a-1578-463b-b603-127cf7974b0e","Type":"ContainerDied","Data":"3d0b8a5ec699b9fe6746b5877988ad3a9aa2e3e3bf5b424416170e4119336467"} Jul 16 09:18:24.450352 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:24.450045 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6fvwz" event={"ID":"f7c3930a-1578-463b-b603-127cf7974b0e","Type":"ContainerDied","Data":"63315bc8bccc92f22728802456fd44effbfca3c66169e45a857250d5da513627"} Jul 16 09:18:24.450352 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:24.450068 2567 scope.go:117] "RemoveContainer" containerID="3d0b8a5ec699b9fe6746b5877988ad3a9aa2e3e3bf5b424416170e4119336467" Jul 16 09:18:24.459156 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:24.459138 2567 scope.go:117] "RemoveContainer" containerID="3d0b8a5ec699b9fe6746b5877988ad3a9aa2e3e3bf5b424416170e4119336467" Jul 16 09:18:24.459413 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:18:24.459392 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d0b8a5ec699b9fe6746b5877988ad3a9aa2e3e3bf5b424416170e4119336467\": container with ID starting with 3d0b8a5ec699b9fe6746b5877988ad3a9aa2e3e3bf5b424416170e4119336467 not found: ID does not exist" containerID="3d0b8a5ec699b9fe6746b5877988ad3a9aa2e3e3bf5b424416170e4119336467" Jul 16 09:18:24.459516 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:24.459422 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d0b8a5ec699b9fe6746b5877988ad3a9aa2e3e3bf5b424416170e4119336467"} err="failed to get container status \"3d0b8a5ec699b9fe6746b5877988ad3a9aa2e3e3bf5b424416170e4119336467\": rpc error: code = NotFound desc = could not find container \"3d0b8a5ec699b9fe6746b5877988ad3a9aa2e3e3bf5b424416170e4119336467\": container with ID starting with 3d0b8a5ec699b9fe6746b5877988ad3a9aa2e3e3bf5b424416170e4119336467 not found: ID does not exist" Jul 16 09:18:24.472059 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:24.472024 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fvwz"] Jul 16 09:18:24.477944 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:24.477916 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6fvwz"] Jul 16 09:18:25.324258 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:25.324226 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7c3930a-1578-463b-b603-127cf7974b0e" path="/var/lib/kubelet/pods/f7c3930a-1578-463b-b603-127cf7974b0e/volumes" Jul 16 09:18:27.330380 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.330345 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn"] Jul 16 09:18:27.330919 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.330715 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c3930a-1578-463b-b603-127cf7974b0e" containerName="limitador" Jul 16 09:18:27.330919 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.330726 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c3930a-1578-463b-b603-127cf7974b0e" containerName="limitador" Jul 16 09:18:27.330919 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.330784 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7c3930a-1578-463b-b603-127cf7974b0e" containerName="limitador" Jul 16 09:18:27.335480 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.335454 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn" Jul 16 09:18:27.338404 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.338376 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 16 09:18:27.338739 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.338721 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 16 09:18:27.338805 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.338726 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 09:18:27.339760 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.339745 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-qrz4f\"" Jul 16 09:18:27.348493 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.348465 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn"] Jul 16 09:18:27.419148 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.419110 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbbmh\" (UniqueName: \"kubernetes.io/projected/59ed8cf2-8dc1-43ed-8256-0cf2bb982be4-kube-api-access-wbbmh\") pod \"postgres-69fb4594cf-pfjkn\" (UID: \"59ed8cf2-8dc1-43ed-8256-0cf2bb982be4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn" Jul 16 09:18:27.419148 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.419164 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/59ed8cf2-8dc1-43ed-8256-0cf2bb982be4-data\") pod \"postgres-69fb4594cf-pfjkn\" (UID: \"59ed8cf2-8dc1-43ed-8256-0cf2bb982be4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn" Jul 16 09:18:27.519864 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.519798 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbbmh\" (UniqueName: \"kubernetes.io/projected/59ed8cf2-8dc1-43ed-8256-0cf2bb982be4-kube-api-access-wbbmh\") pod \"postgres-69fb4594cf-pfjkn\" (UID: \"59ed8cf2-8dc1-43ed-8256-0cf2bb982be4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn" Jul 16 09:18:27.520070 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.519884 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/59ed8cf2-8dc1-43ed-8256-0cf2bb982be4-data\") pod \"postgres-69fb4594cf-pfjkn\" (UID: \"59ed8cf2-8dc1-43ed-8256-0cf2bb982be4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn" Jul 16 09:18:27.520302 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.520281 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/59ed8cf2-8dc1-43ed-8256-0cf2bb982be4-data\") pod \"postgres-69fb4594cf-pfjkn\" (UID: \"59ed8cf2-8dc1-43ed-8256-0cf2bb982be4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn" Jul 16 09:18:27.529509 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.529473 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbbmh\" (UniqueName: \"kubernetes.io/projected/59ed8cf2-8dc1-43ed-8256-0cf2bb982be4-kube-api-access-wbbmh\") pod \"postgres-69fb4594cf-pfjkn\" (UID: \"59ed8cf2-8dc1-43ed-8256-0cf2bb982be4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn" Jul 16 09:18:27.647204 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.647111 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn" Jul 16 09:18:27.783535 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:27.783364 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn"] Jul 16 09:18:27.786430 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:18:27.786400 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ed8cf2_8dc1_43ed_8256_0cf2bb982be4.slice/crio-f18d1a2963b914401763d469739d9e1d8e7577240e5d3af4cffdebdd44dda84c WatchSource:0}: Error finding container f18d1a2963b914401763d469739d9e1d8e7577240e5d3af4cffdebdd44dda84c: Status 404 returned error can't find the container with id f18d1a2963b914401763d469739d9e1d8e7577240e5d3af4cffdebdd44dda84c Jul 16 09:18:28.472791 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:28.472711 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn" event={"ID":"59ed8cf2-8dc1-43ed-8256-0cf2bb982be4","Type":"ContainerStarted","Data":"f18d1a2963b914401763d469739d9e1d8e7577240e5d3af4cffdebdd44dda84c"} Jul 16 09:18:33.501334 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:33.501293 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn" event={"ID":"59ed8cf2-8dc1-43ed-8256-0cf2bb982be4","Type":"ContainerStarted","Data":"9e613cc4499649164617a33a0c4c1c0f8edbc7a616aab25d46e557da88a10797"} Jul 16 09:18:33.501733 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:33.501374 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn" Jul 16 09:18:33.519388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:33.519325 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn" podStartSLOduration=1.6723348059999998 podStartE2EDuration="6.519307631s" totalImagesPullingTime="4.846972825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:18:33.518587098 +0000 UTC m=+788.928135352" watchObservedRunningTime="2026-07-16 09:18:33.519307631 +0000 UTC m=+788.928855925" Jul 16 09:18:39.535594 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:39.535563 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pfjkn" Jul 16 09:18:46.200802 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.200765 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-844954bfc8-64hfp"] Jul 16 09:18:46.207908 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.207883 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-844954bfc8-64hfp" Jul 16 09:18:46.211338 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.211309 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 09:18:46.211338 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.211309 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pmk24\"" Jul 16 09:18:46.211560 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.211414 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 09:18:46.215118 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.215083 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-844954bfc8-64hfp"] Jul 16 09:18:46.308283 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.308248 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2328e63-ff1e-4d2c-8d01-1e3f01acd414-webhook-cert\") pod \"maas-controller-844954bfc8-64hfp\" (UID: \"c2328e63-ff1e-4d2c-8d01-1e3f01acd414\") " pod="opendatahub/maas-controller-844954bfc8-64hfp" Jul 16 09:18:46.308505 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.308340 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r729l\" (UniqueName: \"kubernetes.io/projected/c2328e63-ff1e-4d2c-8d01-1e3f01acd414-kube-api-access-r729l\") pod \"maas-controller-844954bfc8-64hfp\" (UID: \"c2328e63-ff1e-4d2c-8d01-1e3f01acd414\") " pod="opendatahub/maas-controller-844954bfc8-64hfp" Jul 16 09:18:46.368280 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.368247 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66655566cf-tttqk"] Jul 16 09:18:46.371831 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.371796 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66655566cf-tttqk" Jul 16 09:18:46.383451 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.383421 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66655566cf-tttqk"] Jul 16 09:18:46.408954 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.408920 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2328e63-ff1e-4d2c-8d01-1e3f01acd414-webhook-cert\") pod \"maas-controller-844954bfc8-64hfp\" (UID: \"c2328e63-ff1e-4d2c-8d01-1e3f01acd414\") " pod="opendatahub/maas-controller-844954bfc8-64hfp" Jul 16 09:18:46.409129 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.408989 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r729l\" (UniqueName: \"kubernetes.io/projected/c2328e63-ff1e-4d2c-8d01-1e3f01acd414-kube-api-access-r729l\") pod \"maas-controller-844954bfc8-64hfp\" (UID: \"c2328e63-ff1e-4d2c-8d01-1e3f01acd414\") " pod="opendatahub/maas-controller-844954bfc8-64hfp" Jul 16 09:18:46.411388 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.411365 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2328e63-ff1e-4d2c-8d01-1e3f01acd414-webhook-cert\") pod \"maas-controller-844954bfc8-64hfp\" (UID: \"c2328e63-ff1e-4d2c-8d01-1e3f01acd414\") " pod="opendatahub/maas-controller-844954bfc8-64hfp" Jul 16 09:18:46.419167 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.419136 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r729l\" (UniqueName: \"kubernetes.io/projected/c2328e63-ff1e-4d2c-8d01-1e3f01acd414-kube-api-access-r729l\") pod \"maas-controller-844954bfc8-64hfp\" (UID: \"c2328e63-ff1e-4d2c-8d01-1e3f01acd414\") " pod="opendatahub/maas-controller-844954bfc8-64hfp" Jul 16 09:18:46.509671 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.509543 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cbf6440-6d2c-4995-ae26-7e351683c7ba-webhook-cert\") pod \"maas-controller-66655566cf-tttqk\" (UID: \"1cbf6440-6d2c-4995-ae26-7e351683c7ba\") " pod="opendatahub/maas-controller-66655566cf-tttqk" Jul 16 09:18:46.509671 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.509587 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzpdb\" (UniqueName: \"kubernetes.io/projected/1cbf6440-6d2c-4995-ae26-7e351683c7ba-kube-api-access-lzpdb\") pod \"maas-controller-66655566cf-tttqk\" (UID: \"1cbf6440-6d2c-4995-ae26-7e351683c7ba\") " pod="opendatahub/maas-controller-66655566cf-tttqk" Jul 16 09:18:46.520417 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.520385 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-844954bfc8-64hfp" Jul 16 09:18:46.610448 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.610413 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cbf6440-6d2c-4995-ae26-7e351683c7ba-webhook-cert\") pod \"maas-controller-66655566cf-tttqk\" (UID: \"1cbf6440-6d2c-4995-ae26-7e351683c7ba\") " pod="opendatahub/maas-controller-66655566cf-tttqk" Jul 16 09:18:46.610634 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.610453 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzpdb\" (UniqueName: \"kubernetes.io/projected/1cbf6440-6d2c-4995-ae26-7e351683c7ba-kube-api-access-lzpdb\") pod \"maas-controller-66655566cf-tttqk\" (UID: \"1cbf6440-6d2c-4995-ae26-7e351683c7ba\") " pod="opendatahub/maas-controller-66655566cf-tttqk" Jul 16 09:18:46.613051 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.613022 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cbf6440-6d2c-4995-ae26-7e351683c7ba-webhook-cert\") pod \"maas-controller-66655566cf-tttqk\" (UID: \"1cbf6440-6d2c-4995-ae26-7e351683c7ba\") " pod="opendatahub/maas-controller-66655566cf-tttqk" Jul 16 09:18:46.619337 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.619313 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzpdb\" (UniqueName: \"kubernetes.io/projected/1cbf6440-6d2c-4995-ae26-7e351683c7ba-kube-api-access-lzpdb\") pod \"maas-controller-66655566cf-tttqk\" (UID: \"1cbf6440-6d2c-4995-ae26-7e351683c7ba\") " pod="opendatahub/maas-controller-66655566cf-tttqk" Jul 16 09:18:46.656986 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.656960 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-844954bfc8-64hfp"] Jul 16 09:18:46.658805 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:18:46.658772 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2328e63_ff1e_4d2c_8d01_1e3f01acd414.slice/crio-1f46588939286d5b3fed7bf3823c86ffaf661750b94c545fdce2f0a927ca679f WatchSource:0}: Error finding container 1f46588939286d5b3fed7bf3823c86ffaf661750b94c545fdce2f0a927ca679f: Status 404 returned error can't find the container with id 1f46588939286d5b3fed7bf3823c86ffaf661750b94c545fdce2f0a927ca679f Jul 16 09:18:46.682535 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.682501 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66655566cf-tttqk" Jul 16 09:18:46.818090 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:46.818066 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66655566cf-tttqk"] Jul 16 09:18:46.820680 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:18:46.820656 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cbf6440_6d2c_4995_ae26_7e351683c7ba.slice/crio-0225094902d9e439abca2a4692cbb4b3d7ee9a0be4ad384cdaf85c57b3c20ff3 WatchSource:0}: Error finding container 0225094902d9e439abca2a4692cbb4b3d7ee9a0be4ad384cdaf85c57b3c20ff3: Status 404 returned error can't find the container with id 0225094902d9e439abca2a4692cbb4b3d7ee9a0be4ad384cdaf85c57b3c20ff3 Jul 16 09:18:47.573874 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:47.573832 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-844954bfc8-64hfp" event={"ID":"c2328e63-ff1e-4d2c-8d01-1e3f01acd414","Type":"ContainerStarted","Data":"1f46588939286d5b3fed7bf3823c86ffaf661750b94c545fdce2f0a927ca679f"} Jul 16 09:18:47.578114 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:47.578078 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66655566cf-tttqk" event={"ID":"1cbf6440-6d2c-4995-ae26-7e351683c7ba","Type":"ContainerStarted","Data":"0225094902d9e439abca2a4692cbb4b3d7ee9a0be4ad384cdaf85c57b3c20ff3"} Jul 16 09:18:50.596467 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:50.596431 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-844954bfc8-64hfp" event={"ID":"c2328e63-ff1e-4d2c-8d01-1e3f01acd414","Type":"ContainerStarted","Data":"0c6d45bdd5feb7715f1dde8eab93f9f3f4dd9d755e40cf1d65ba0a658a8f7ef6"} Jul 16 09:18:50.596987 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:50.596504 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-844954bfc8-64hfp" Jul 16 09:18:50.597980 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:50.597952 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66655566cf-tttqk" event={"ID":"1cbf6440-6d2c-4995-ae26-7e351683c7ba","Type":"ContainerStarted","Data":"469788a981149e6fcc2fe8ae478cdc1139f12d8fb3671ae7314e5a95d70dd66c"} Jul 16 09:18:50.598115 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:50.598086 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66655566cf-tttqk" Jul 16 09:18:50.617125 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:50.617078 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-844954bfc8-64hfp" podStartSLOduration=1.760678521 podStartE2EDuration="4.61706222s" totalImagesPullingTime="2.856383699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:18:50.615613002 +0000 UTC m=+806.025161275" watchObservedRunningTime="2026-07-16 09:18:50.61706222 +0000 UTC m=+806.026610478" Jul 16 09:18:50.634738 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:18:50.634680 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66655566cf-tttqk" podStartSLOduration=1.94778739 podStartE2EDuration="4.634661681s" totalImagesPullingTime="2.686874291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:18:50.634046208 +0000 UTC m=+806.043594491" watchObservedRunningTime="2026-07-16 09:18:50.634661681 +0000 UTC m=+806.044209940" Jul 16 09:19:01.605313 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:01.605270 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66655566cf-tttqk" Jul 16 09:19:01.605740 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:01.605365 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-844954bfc8-64hfp" Jul 16 09:19:01.685642 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:01.685611 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-844954bfc8-64hfp"] Jul 16 09:19:01.685804 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:01.685785 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-844954bfc8-64hfp" podUID="c2328e63-ff1e-4d2c-8d01-1e3f01acd414" containerName="manager" containerID="cri-o://0c6d45bdd5feb7715f1dde8eab93f9f3f4dd9d755e40cf1d65ba0a658a8f7ef6" gracePeriod=30 Jul 16 09:19:01.938292 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:01.938265 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-844954bfc8-64hfp" Jul 16 09:19:01.958995 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:01.958951 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r729l\" (UniqueName: \"kubernetes.io/projected/c2328e63-ff1e-4d2c-8d01-1e3f01acd414-kube-api-access-r729l\") pod \"c2328e63-ff1e-4d2c-8d01-1e3f01acd414\" (UID: \"c2328e63-ff1e-4d2c-8d01-1e3f01acd414\") " Jul 16 09:19:01.959180 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:01.959056 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2328e63-ff1e-4d2c-8d01-1e3f01acd414-webhook-cert\") pod \"c2328e63-ff1e-4d2c-8d01-1e3f01acd414\" (UID: \"c2328e63-ff1e-4d2c-8d01-1e3f01acd414\") " Jul 16 09:19:01.961468 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:01.961439 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2328e63-ff1e-4d2c-8d01-1e3f01acd414-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c2328e63-ff1e-4d2c-8d01-1e3f01acd414" (UID: "c2328e63-ff1e-4d2c-8d01-1e3f01acd414"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:19:01.961468 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:01.961449 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2328e63-ff1e-4d2c-8d01-1e3f01acd414-kube-api-access-r729l" (OuterVolumeSpecName: "kube-api-access-r729l") pod "c2328e63-ff1e-4d2c-8d01-1e3f01acd414" (UID: "c2328e63-ff1e-4d2c-8d01-1e3f01acd414"). InnerVolumeSpecName "kube-api-access-r729l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 09:19:02.060273 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:02.060233 2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2328e63-ff1e-4d2c-8d01-1e3f01acd414-webhook-cert\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:19:02.060273 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:02.060268 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r729l\" (UniqueName: \"kubernetes.io/projected/c2328e63-ff1e-4d2c-8d01-1e3f01acd414-kube-api-access-r729l\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:19:02.658897 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:02.658862 2567 generic.go:356] "Generic (PLEG): container finished" podID="c2328e63-ff1e-4d2c-8d01-1e3f01acd414" containerID="0c6d45bdd5feb7715f1dde8eab93f9f3f4dd9d755e40cf1d65ba0a658a8f7ef6" exitCode=0 Jul 16 09:19:02.659296 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:02.658919 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-844954bfc8-64hfp" Jul 16 09:19:02.659296 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:02.658953 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-844954bfc8-64hfp" event={"ID":"c2328e63-ff1e-4d2c-8d01-1e3f01acd414","Type":"ContainerDied","Data":"0c6d45bdd5feb7715f1dde8eab93f9f3f4dd9d755e40cf1d65ba0a658a8f7ef6"} Jul 16 09:19:02.659296 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:02.659008 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-844954bfc8-64hfp" event={"ID":"c2328e63-ff1e-4d2c-8d01-1e3f01acd414","Type":"ContainerDied","Data":"1f46588939286d5b3fed7bf3823c86ffaf661750b94c545fdce2f0a927ca679f"} Jul 16 09:19:02.659296 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:02.659038 2567 scope.go:117] "RemoveContainer" containerID="0c6d45bdd5feb7715f1dde8eab93f9f3f4dd9d755e40cf1d65ba0a658a8f7ef6" Jul 16 09:19:02.668258 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:02.668240 2567 scope.go:117] "RemoveContainer" containerID="0c6d45bdd5feb7715f1dde8eab93f9f3f4dd9d755e40cf1d65ba0a658a8f7ef6" Jul 16 09:19:02.668549 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:19:02.668530 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c6d45bdd5feb7715f1dde8eab93f9f3f4dd9d755e40cf1d65ba0a658a8f7ef6\": container with ID starting with 0c6d45bdd5feb7715f1dde8eab93f9f3f4dd9d755e40cf1d65ba0a658a8f7ef6 not found: ID does not exist" containerID="0c6d45bdd5feb7715f1dde8eab93f9f3f4dd9d755e40cf1d65ba0a658a8f7ef6" Jul 16 09:19:02.668621 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:02.668564 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c6d45bdd5feb7715f1dde8eab93f9f3f4dd9d755e40cf1d65ba0a658a8f7ef6"} err="failed to get container status \"0c6d45bdd5feb7715f1dde8eab93f9f3f4dd9d755e40cf1d65ba0a658a8f7ef6\": rpc error: code = NotFound desc = could not find container \"0c6d45bdd5feb7715f1dde8eab93f9f3f4dd9d755e40cf1d65ba0a658a8f7ef6\": container with ID starting with 0c6d45bdd5feb7715f1dde8eab93f9f3f4dd9d755e40cf1d65ba0a658a8f7ef6 not found: ID does not exist" Jul 16 09:19:02.681320 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:02.681282 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-844954bfc8-64hfp"] Jul 16 09:19:02.684984 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:02.684960 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-844954bfc8-64hfp"] Jul 16 09:19:03.324910 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:03.324879 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2328e63-ff1e-4d2c-8d01-1e3f01acd414" path="/var/lib/kubelet/pods/c2328e63-ff1e-4d2c-8d01-1e3f01acd414/volumes" Jul 16 09:19:22.153183 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.153144 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4"] Jul 16 09:19:22.153656 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.153568 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2328e63-ff1e-4d2c-8d01-1e3f01acd414" containerName="manager" Jul 16 09:19:22.153656 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.153579 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2328e63-ff1e-4d2c-8d01-1e3f01acd414" containerName="manager" Jul 16 09:19:22.153656 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.153640 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2328e63-ff1e-4d2c-8d01-1e3f01acd414" containerName="manager" Jul 16 09:19:22.156639 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.156623 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" Jul 16 09:19:22.159440 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.159413 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 16 09:19:22.159916 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.159900 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vp4zf\"" Jul 16 09:19:22.168339 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.168314 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4"] Jul 16 09:19:22.248395 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.248351 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a4dd53-4139-4e35-a1fd-6e69388fd55c-maas-api-tls\") pod \"maas-api-7fb9f65958-bb5t4\" (UID: \"f2a4dd53-4139-4e35-a1fd-6e69388fd55c\") " pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" Jul 16 09:19:22.248395 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.248390 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a4dd53-4139-4e35-a1fd-6e69388fd55c-openshift-service-ca\") pod \"maas-api-7fb9f65958-bb5t4\" (UID: \"f2a4dd53-4139-4e35-a1fd-6e69388fd55c\") " pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" Jul 16 09:19:22.248656 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.248545 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x56v\" (UniqueName: \"kubernetes.io/projected/f2a4dd53-4139-4e35-a1fd-6e69388fd55c-kube-api-access-9x56v\") pod \"maas-api-7fb9f65958-bb5t4\" (UID: \"f2a4dd53-4139-4e35-a1fd-6e69388fd55c\") " pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" Jul 16 09:19:22.349916 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.349874 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x56v\" (UniqueName: \"kubernetes.io/projected/f2a4dd53-4139-4e35-a1fd-6e69388fd55c-kube-api-access-9x56v\") pod \"maas-api-7fb9f65958-bb5t4\" (UID: \"f2a4dd53-4139-4e35-a1fd-6e69388fd55c\") " pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" Jul 16 09:19:22.349916 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.349922 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a4dd53-4139-4e35-a1fd-6e69388fd55c-maas-api-tls\") pod \"maas-api-7fb9f65958-bb5t4\" (UID: \"f2a4dd53-4139-4e35-a1fd-6e69388fd55c\") " pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" Jul 16 09:19:22.350121 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.349939 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a4dd53-4139-4e35-a1fd-6e69388fd55c-openshift-service-ca\") pod \"maas-api-7fb9f65958-bb5t4\" (UID: \"f2a4dd53-4139-4e35-a1fd-6e69388fd55c\") " pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" Jul 16 09:19:22.350121 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:19:22.350049 2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 16 09:19:22.350188 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:19:22.350129 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2a4dd53-4139-4e35-a1fd-6e69388fd55c-maas-api-tls podName:f2a4dd53-4139-4e35-a1fd-6e69388fd55c nodeName:}" failed. No retries permitted until 2026-07-16 09:19:22.85010655 +0000 UTC m=+838.259654800 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f2a4dd53-4139-4e35-a1fd-6e69388fd55c-maas-api-tls") pod "maas-api-7fb9f65958-bb5t4" (UID: "f2a4dd53-4139-4e35-a1fd-6e69388fd55c") : secret "maas-api-serving-cert" not found Jul 16 09:19:22.350570 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.350553 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2a4dd53-4139-4e35-a1fd-6e69388fd55c-openshift-service-ca\") pod \"maas-api-7fb9f65958-bb5t4\" (UID: \"f2a4dd53-4139-4e35-a1fd-6e69388fd55c\") " pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" Jul 16 09:19:22.360658 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.360625 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x56v\" (UniqueName: \"kubernetes.io/projected/f2a4dd53-4139-4e35-a1fd-6e69388fd55c-kube-api-access-9x56v\") pod \"maas-api-7fb9f65958-bb5t4\" (UID: \"f2a4dd53-4139-4e35-a1fd-6e69388fd55c\") " pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" Jul 16 09:19:22.681211 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.681169 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r"] Jul 16 09:19:22.685212 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.685193 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r" Jul 16 09:19:22.689131 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.689109 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-gx9m5\"" Jul 16 09:19:22.689288 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.689177 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 16 09:19:22.696593 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.696569 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r"] Jul 16 09:19:22.713205 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.713175 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-r6h5h"] Jul 16 09:19:22.716646 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.716629 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-r6h5h" Jul 16 09:19:22.725536 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.725507 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-r6h5h"] Jul 16 09:19:22.753601 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.753569 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhvl9\" (UniqueName: \"kubernetes.io/projected/9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd-kube-api-access-lhvl9\") pod \"payload-pre-processing-74584bb9f8-jxj7r\" (UID: \"9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r" Jul 16 09:19:22.753786 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.753630 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-jxj7r\" (UID: \"9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r" Jul 16 09:19:22.854968 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.854922 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhvl9\" (UniqueName: \"kubernetes.io/projected/9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd-kube-api-access-lhvl9\") pod \"payload-pre-processing-74584bb9f8-jxj7r\" (UID: \"9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r" Jul 16 09:19:22.855163 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.854986 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd9s6\" (UniqueName: \"kubernetes.io/projected/22460db3-fa52-4f5c-a034-44b273beeae9-kube-api-access-sd9s6\") pod \"payload-processing-f9b8d9b9b-r6h5h\" (UID: \"22460db3-fa52-4f5c-a034-44b273beeae9\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-r6h5h" Jul 16 09:19:22.855163 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.855045 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a4dd53-4139-4e35-a1fd-6e69388fd55c-maas-api-tls\") pod \"maas-api-7fb9f65958-bb5t4\" (UID: \"f2a4dd53-4139-4e35-a1fd-6e69388fd55c\") " pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" Jul 16 09:19:22.855163 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.855094 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-jxj7r\" (UID: \"9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r" Jul 16 09:19:22.855350 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.855201 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22460db3-fa52-4f5c-a034-44b273beeae9-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-r6h5h\" (UID: \"22460db3-fa52-4f5c-a034-44b273beeae9\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-r6h5h" Jul 16 09:19:22.855833 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.855804 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-jxj7r\" (UID: \"9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r" Jul 16 09:19:22.857522 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.857501 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a4dd53-4139-4e35-a1fd-6e69388fd55c-maas-api-tls\") pod \"maas-api-7fb9f65958-bb5t4\" (UID: \"f2a4dd53-4139-4e35-a1fd-6e69388fd55c\") " pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" Jul 16 09:19:22.864217 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.864192 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhvl9\" (UniqueName: \"kubernetes.io/projected/9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd-kube-api-access-lhvl9\") pod \"payload-pre-processing-74584bb9f8-jxj7r\" (UID: \"9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r" Jul 16 09:19:22.956410 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.956318 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22460db3-fa52-4f5c-a034-44b273beeae9-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-r6h5h\" (UID: \"22460db3-fa52-4f5c-a034-44b273beeae9\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-r6h5h" Jul 16 09:19:22.956410 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.956395 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd9s6\" (UniqueName: \"kubernetes.io/projected/22460db3-fa52-4f5c-a034-44b273beeae9-kube-api-access-sd9s6\") pod \"payload-processing-f9b8d9b9b-r6h5h\" (UID: \"22460db3-fa52-4f5c-a034-44b273beeae9\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-r6h5h" Jul 16 09:19:22.957244 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.957224 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22460db3-fa52-4f5c-a034-44b273beeae9-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-r6h5h\" (UID: \"22460db3-fa52-4f5c-a034-44b273beeae9\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-r6h5h" Jul 16 09:19:22.968590 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.968555 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd9s6\" (UniqueName: \"kubernetes.io/projected/22460db3-fa52-4f5c-a034-44b273beeae9-kube-api-access-sd9s6\") pod \"payload-processing-f9b8d9b9b-r6h5h\" (UID: \"22460db3-fa52-4f5c-a034-44b273beeae9\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-r6h5h" Jul 16 09:19:22.996496 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:22.996461 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r" Jul 16 09:19:23.027786 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:23.027757 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-r6h5h" Jul 16 09:19:23.068661 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:23.068533 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" Jul 16 09:19:23.158319 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:23.158270 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r"] Jul 16 09:19:23.161476 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:19:23.161431 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f67aea2_02a5_45eb_bb5a_17c92d0bf8fd.slice/crio-fe224a3e5d27e21c7b149d1e6f3ff1da67173e9f757c03386696bb89432cb0ed WatchSource:0}: Error finding container fe224a3e5d27e21c7b149d1e6f3ff1da67173e9f757c03386696bb89432cb0ed: Status 404 returned error can't find the container with id fe224a3e5d27e21c7b149d1e6f3ff1da67173e9f757c03386696bb89432cb0ed Jul 16 09:19:23.195494 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:23.195468 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-r6h5h"] Jul 16 09:19:23.197856 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:19:23.197791 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22460db3_fa52_4f5c_a034_44b273beeae9.slice/crio-790ae0e0b752e7d151e4dca7d3f8d254761b5da264abb29605eb9fd056194068 WatchSource:0}: Error finding container 790ae0e0b752e7d151e4dca7d3f8d254761b5da264abb29605eb9fd056194068: Status 404 returned error can't find the container with id 790ae0e0b752e7d151e4dca7d3f8d254761b5da264abb29605eb9fd056194068 Jul 16 09:19:23.240517 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:23.240487 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4"] Jul 16 09:19:23.243276 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:19:23.243241 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a4dd53_4139_4e35_a1fd_6e69388fd55c.slice/crio-c93ffdc4e0694e7f999c2e2d44ded221ac7a37f7842927d7028b893a6aa46c99 WatchSource:0}: Error finding container c93ffdc4e0694e7f999c2e2d44ded221ac7a37f7842927d7028b893a6aa46c99: Status 404 returned error can't find the container with id c93ffdc4e0694e7f999c2e2d44ded221ac7a37f7842927d7028b893a6aa46c99 Jul 16 09:19:23.762248 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:23.762210 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" event={"ID":"f2a4dd53-4139-4e35-a1fd-6e69388fd55c","Type":"ContainerStarted","Data":"c93ffdc4e0694e7f999c2e2d44ded221ac7a37f7842927d7028b893a6aa46c99"} Jul 16 09:19:23.763426 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:23.763396 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r" event={"ID":"9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd","Type":"ContainerStarted","Data":"fe224a3e5d27e21c7b149d1e6f3ff1da67173e9f757c03386696bb89432cb0ed"} Jul 16 09:19:23.764610 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:23.764588 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-r6h5h" event={"ID":"22460db3-fa52-4f5c-a034-44b273beeae9","Type":"ContainerStarted","Data":"790ae0e0b752e7d151e4dca7d3f8d254761b5da264abb29605eb9fd056194068"} Jul 16 09:19:25.776858 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:25.776794 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-r6h5h" event={"ID":"22460db3-fa52-4f5c-a034-44b273beeae9","Type":"ContainerStarted","Data":"9e0959ca7b01669d1528536260fc30945a19023bb7ebf1ca671212478e965828"} Jul 16 09:19:25.777335 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:25.777028 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-r6h5h" Jul 16 09:19:25.778382 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:25.778358 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" event={"ID":"f2a4dd53-4139-4e35-a1fd-6e69388fd55c","Type":"ContainerStarted","Data":"0dc7337b28a6fa207ad22ef253d3ee5c8eb20b5f8bd18c415c818b6a335fdb1b"} Jul 16 09:19:25.778512 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:25.778407 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" Jul 16 09:19:25.779720 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:25.779699 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r" event={"ID":"9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd","Type":"ContainerStarted","Data":"0118e8a21a782b18aba7a0db8691e388b19c171936aa8441a1dea0261cb5f6a1"} Jul 16 09:19:25.779847 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:25.779832 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r" Jul 16 09:19:25.793716 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:25.793670 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-r6h5h" podStartSLOduration=2.104770076 podStartE2EDuration="3.793654773s" totalImagesPullingTime="1.688884697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:19:25.791146861 +0000 UTC m=+841.200695146" watchObservedRunningTime="2026-07-16 09:19:25.793654773 +0000 UTC m=+841.203203031" Jul 16 09:19:25.810040 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:25.809986 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" podStartSLOduration=2.165714309 podStartE2EDuration="3.809964992s" totalImagesPullingTime="1.644250683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:19:25.806663601 +0000 UTC m=+841.216211861" watchObservedRunningTime="2026-07-16 09:19:25.809964992 +0000 UTC m=+841.219513254" Jul 16 09:19:25.822746 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:25.822695 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r" podStartSLOduration=2.095460999 podStartE2EDuration="3.822676956s" totalImagesPullingTime="1.727215957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:19:25.821897613 +0000 UTC m=+841.231445903" watchObservedRunningTime="2026-07-16 09:19:25.822676956 +0000 UTC m=+841.232225215" Jul 16 09:19:31.790559 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:31.790523 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7fb9f65958-bb5t4" Jul 16 09:19:36.786546 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:36.786513 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-r6h5h" Jul 16 09:19:36.786951 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:36.786601 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jxj7r" Jul 16 09:19:43.461989 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.461953 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm"] Jul 16 09:19:43.466101 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.466077 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.470176 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.469931 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5qshb\"" Jul 16 09:19:43.470176 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.469983 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 09:19:43.470176 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.470045 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 16 09:19:43.470176 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.469946 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 09:19:43.478617 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.478592 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm"] Jul 16 09:19:43.553070 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.553027 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a823d7f7-a285-4952-bb3b-48eada039386-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.553250 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.553110 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnpnc\" (UniqueName: \"kubernetes.io/projected/a823d7f7-a285-4952-bb3b-48eada039386-kube-api-access-lnpnc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.553250 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.553154 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a823d7f7-a285-4952-bb3b-48eada039386-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.553389 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.553255 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a823d7f7-a285-4952-bb3b-48eada039386-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.553389 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.553302 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a823d7f7-a285-4952-bb3b-48eada039386-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.553389 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.553350 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a823d7f7-a285-4952-bb3b-48eada039386-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.654439 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.654400 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a823d7f7-a285-4952-bb3b-48eada039386-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.654653 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.654463 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnpnc\" (UniqueName: \"kubernetes.io/projected/a823d7f7-a285-4952-bb3b-48eada039386-kube-api-access-lnpnc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.654653 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.654511 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a823d7f7-a285-4952-bb3b-48eada039386-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.654653 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.654555 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a823d7f7-a285-4952-bb3b-48eada039386-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.654653 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.654589 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a823d7f7-a285-4952-bb3b-48eada039386-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.654653 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.654623 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a823d7f7-a285-4952-bb3b-48eada039386-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.654972 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.654933 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a823d7f7-a285-4952-bb3b-48eada039386-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.654972 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.654959 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a823d7f7-a285-4952-bb3b-48eada039386-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.655087 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.655035 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a823d7f7-a285-4952-bb3b-48eada039386-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.657041 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.657018 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a823d7f7-a285-4952-bb3b-48eada039386-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.657183 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.657167 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a823d7f7-a285-4952-bb3b-48eada039386-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.663028 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.663006 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnpnc\" (UniqueName: \"kubernetes.io/projected/a823d7f7-a285-4952-bb3b-48eada039386-kube-api-access-lnpnc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm\" (UID: \"a823d7f7-a285-4952-bb3b-48eada039386\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.778925 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.778889 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:43.921962 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:43.921932 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm"] Jul 16 09:19:43.923881 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:19:43.923853 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda823d7f7_a285_4952_bb3b_48eada039386.slice/crio-435adc848d245c12d3283f6634fa5247dcb31f604022be7123413f6a760c8e96 WatchSource:0}: Error finding container 435adc848d245c12d3283f6634fa5247dcb31f604022be7123413f6a760c8e96: Status 404 returned error can't find the container with id 435adc848d245c12d3283f6634fa5247dcb31f604022be7123413f6a760c8e96 Jul 16 09:19:44.882941 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:44.882867 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" event={"ID":"a823d7f7-a285-4952-bb3b-48eada039386","Type":"ContainerStarted","Data":"435adc848d245c12d3283f6634fa5247dcb31f604022be7123413f6a760c8e96"} Jul 16 09:19:45.888937 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:45.888899 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" event={"ID":"a823d7f7-a285-4952-bb3b-48eada039386","Type":"ContainerStarted","Data":"665e41ee6348c5592af1700cab99f3408ce21ea9825b70df4a60d9ee1cadd04a"} Jul 16 09:19:45.909030 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:45.908977 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" podStartSLOduration=1.677306907 podStartE2EDuration="2.908960312s" totalImagesPullingTime="1.231653405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:19:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:19:45.907959409 +0000 UTC m=+861.317507680" watchObservedRunningTime="2026-07-16 09:19:45.908960312 +0000 UTC m=+861.318508582" Jul 16 09:19:53.082673 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.082630 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl"] Jul 16 09:19:53.086966 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.086942 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.089917 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.089883 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 16 09:19:53.094732 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.094700 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl"] Jul 16 09:19:53.152428 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.152363 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1961b8e6-97be-444d-9aa4-127ee1126279-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.152428 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.152426 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1961b8e6-97be-444d-9aa4-127ee1126279-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.152690 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.152571 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1961b8e6-97be-444d-9aa4-127ee1126279-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.152690 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.152651 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1961b8e6-97be-444d-9aa4-127ee1126279-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.152807 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.152701 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1961b8e6-97be-444d-9aa4-127ee1126279-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.152807 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.152729 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7gkv\" (UniqueName: \"kubernetes.io/projected/1961b8e6-97be-444d-9aa4-127ee1126279-kube-api-access-p7gkv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.253524 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.253482 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1961b8e6-97be-444d-9aa4-127ee1126279-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.253876 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.253541 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7gkv\" (UniqueName: \"kubernetes.io/projected/1961b8e6-97be-444d-9aa4-127ee1126279-kube-api-access-p7gkv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.253876 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.253590 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1961b8e6-97be-444d-9aa4-127ee1126279-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.253876 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.253612 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1961b8e6-97be-444d-9aa4-127ee1126279-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.253876 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.253682 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1961b8e6-97be-444d-9aa4-127ee1126279-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.253876 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.253757 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1961b8e6-97be-444d-9aa4-127ee1126279-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.254191 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.254050 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1961b8e6-97be-444d-9aa4-127ee1126279-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.254191 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.254095 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1961b8e6-97be-444d-9aa4-127ee1126279-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.254191 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.254141 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1961b8e6-97be-444d-9aa4-127ee1126279-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.255811 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.255784 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1961b8e6-97be-444d-9aa4-127ee1126279-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.256083 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.256065 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1961b8e6-97be-444d-9aa4-127ee1126279-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.263282 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.263256 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7gkv\" (UniqueName: \"kubernetes.io/projected/1961b8e6-97be-444d-9aa4-127ee1126279-kube-api-access-p7gkv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dwlpl\" (UID: \"1961b8e6-97be-444d-9aa4-127ee1126279\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.399393 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.399299 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:19:53.536873 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.536846 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl"] Jul 16 09:19:53.538981 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:19:53.538948 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1961b8e6_97be_444d_9aa4_127ee1126279.slice/crio-e616a005ac0c50ca0d6941c457a5089c179517c8bdc4d3dcb9fef52d4cea42f3 WatchSource:0}: Error finding container e616a005ac0c50ca0d6941c457a5089c179517c8bdc4d3dcb9fef52d4cea42f3: Status 404 returned error can't find the container with id e616a005ac0c50ca0d6941c457a5089c179517c8bdc4d3dcb9fef52d4cea42f3 Jul 16 09:19:53.779494 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.779459 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:53.779653 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.779520 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:53.792712 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.792687 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:53.933440 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.933397 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" event={"ID":"1961b8e6-97be-444d-9aa4-127ee1126279","Type":"ContainerStarted","Data":"5bccde18e81669012d28a26bc5b11d5ab27fac4a75c77921ee9332992535ae9c"} Jul 16 09:19:53.933636 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.933466 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" event={"ID":"1961b8e6-97be-444d-9aa4-127ee1126279","Type":"ContainerStarted","Data":"e616a005ac0c50ca0d6941c457a5089c179517c8bdc4d3dcb9fef52d4cea42f3"} Jul 16 09:19:53.947703 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.947664 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm" Jul 16 09:19:53.954589 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:19:53.954537 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" podStartSLOduration=0.759492823 podStartE2EDuration="954.524076ms" totalImagesPullingTime="195.031253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:19:53.953956281 +0000 UTC m=+869.363504556" watchObservedRunningTime="2026-07-16 09:19:53.954524076 +0000 UTC m=+869.364072335" Jul 16 09:20:03.400292 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:20:03.400250 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:20:03.400735 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:20:03.400323 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:20:03.412933 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:20:03.412894 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:20:03.994584 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:20:03.994554 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dwlpl" Jul 16 09:20:25.144598 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:20:25.144570 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:20:25.145536 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:20:25.145518 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:23:48.212315 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:48.212275 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66655566cf-tttqk"] Jul 16 09:23:48.212951 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:48.212516 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-66655566cf-tttqk" podUID="1cbf6440-6d2c-4995-ae26-7e351683c7ba" containerName="manager" containerID="cri-o://469788a981149e6fcc2fe8ae478cdc1139f12d8fb3671ae7314e5a95d70dd66c" gracePeriod=30 Jul 16 09:23:48.467171 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:48.467106 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66655566cf-tttqk" Jul 16 09:23:48.536639 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:48.536603 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzpdb\" (UniqueName: \"kubernetes.io/projected/1cbf6440-6d2c-4995-ae26-7e351683c7ba-kube-api-access-lzpdb\") pod \"1cbf6440-6d2c-4995-ae26-7e351683c7ba\" (UID: \"1cbf6440-6d2c-4995-ae26-7e351683c7ba\") " Jul 16 09:23:48.536802 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:48.536728 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cbf6440-6d2c-4995-ae26-7e351683c7ba-webhook-cert\") pod \"1cbf6440-6d2c-4995-ae26-7e351683c7ba\" (UID: \"1cbf6440-6d2c-4995-ae26-7e351683c7ba\") " Jul 16 09:23:48.538657 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:48.538625 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cbf6440-6d2c-4995-ae26-7e351683c7ba-kube-api-access-lzpdb" (OuterVolumeSpecName: "kube-api-access-lzpdb") pod "1cbf6440-6d2c-4995-ae26-7e351683c7ba" (UID: "1cbf6440-6d2c-4995-ae26-7e351683c7ba"). InnerVolumeSpecName "kube-api-access-lzpdb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 09:23:48.538771 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:48.538697 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cbf6440-6d2c-4995-ae26-7e351683c7ba-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1cbf6440-6d2c-4995-ae26-7e351683c7ba" (UID: "1cbf6440-6d2c-4995-ae26-7e351683c7ba"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 09:23:48.638252 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:48.638214 2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cbf6440-6d2c-4995-ae26-7e351683c7ba-webhook-cert\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:23:48.638252 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:48.638245 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzpdb\" (UniqueName: \"kubernetes.io/projected/1cbf6440-6d2c-4995-ae26-7e351683c7ba-kube-api-access-lzpdb\") on node \"ip-10-0-134-16.ec2.internal\" DevicePath \"\"" Jul 16 09:23:49.125775 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.125741 2567 generic.go:356] "Generic (PLEG): container finished" podID="1cbf6440-6d2c-4995-ae26-7e351683c7ba" containerID="469788a981149e6fcc2fe8ae478cdc1139f12d8fb3671ae7314e5a95d70dd66c" exitCode=0 Jul 16 09:23:49.125988 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.125782 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66655566cf-tttqk" Jul 16 09:23:49.125988 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.125840 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66655566cf-tttqk" event={"ID":"1cbf6440-6d2c-4995-ae26-7e351683c7ba","Type":"ContainerDied","Data":"469788a981149e6fcc2fe8ae478cdc1139f12d8fb3671ae7314e5a95d70dd66c"} Jul 16 09:23:49.125988 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.125891 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66655566cf-tttqk" event={"ID":"1cbf6440-6d2c-4995-ae26-7e351683c7ba","Type":"ContainerDied","Data":"0225094902d9e439abca2a4692cbb4b3d7ee9a0be4ad384cdaf85c57b3c20ff3"} Jul 16 09:23:49.125988 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.125916 2567 scope.go:117] "RemoveContainer" containerID="469788a981149e6fcc2fe8ae478cdc1139f12d8fb3671ae7314e5a95d70dd66c" Jul 16 09:23:49.138160 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.138133 2567 scope.go:117] "RemoveContainer" containerID="469788a981149e6fcc2fe8ae478cdc1139f12d8fb3671ae7314e5a95d70dd66c" Jul 16 09:23:49.138503 ip-10-0-134-16 kubenswrapper[2567]: E0716 09:23:49.138480 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"469788a981149e6fcc2fe8ae478cdc1139f12d8fb3671ae7314e5a95d70dd66c\": container with ID starting with 469788a981149e6fcc2fe8ae478cdc1139f12d8fb3671ae7314e5a95d70dd66c not found: ID does not exist" containerID="469788a981149e6fcc2fe8ae478cdc1139f12d8fb3671ae7314e5a95d70dd66c" Jul 16 09:23:49.138557 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.138514 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"469788a981149e6fcc2fe8ae478cdc1139f12d8fb3671ae7314e5a95d70dd66c"} err="failed to get container status \"469788a981149e6fcc2fe8ae478cdc1139f12d8fb3671ae7314e5a95d70dd66c\": rpc error: code = NotFound desc = could not find container \"469788a981149e6fcc2fe8ae478cdc1139f12d8fb3671ae7314e5a95d70dd66c\": container with ID starting with 469788a981149e6fcc2fe8ae478cdc1139f12d8fb3671ae7314e5a95d70dd66c not found: ID does not exist" Jul 16 09:23:49.147683 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.147658 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66655566cf-tttqk"] Jul 16 09:23:49.151560 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.151537 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-66655566cf-tttqk"] Jul 16 09:23:49.324727 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.324690 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cbf6440-6d2c-4995-ae26-7e351683c7ba" path="/var/lib/kubelet/pods/1cbf6440-6d2c-4995-ae26-7e351683c7ba/volumes" Jul 16 09:23:49.926798 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.926762 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66655566cf-nfw9v"] Jul 16 09:23:49.927197 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.927184 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cbf6440-6d2c-4995-ae26-7e351683c7ba" containerName="manager" Jul 16 09:23:49.927247 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.927198 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cbf6440-6d2c-4995-ae26-7e351683c7ba" containerName="manager" Jul 16 09:23:49.927282 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.927268 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cbf6440-6d2c-4995-ae26-7e351683c7ba" containerName="manager" Jul 16 09:23:49.931687 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.931669 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66655566cf-nfw9v" Jul 16 09:23:49.935575 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.935544 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 09:23:49.935575 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.935551 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 09:23:49.935803 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.935617 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pmk24\"" Jul 16 09:23:49.940344 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:49.940317 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66655566cf-nfw9v"] Jul 16 09:23:50.054363 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:50.054319 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6-webhook-cert\") pod \"maas-controller-66655566cf-nfw9v\" (UID: \"9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6\") " pod="opendatahub/maas-controller-66655566cf-nfw9v" Jul 16 09:23:50.054547 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:50.054471 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmrn2\" (UniqueName: \"kubernetes.io/projected/9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6-kube-api-access-rmrn2\") pod \"maas-controller-66655566cf-nfw9v\" (UID: \"9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6\") " pod="opendatahub/maas-controller-66655566cf-nfw9v" Jul 16 09:23:50.155319 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:50.155273 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6-webhook-cert\") pod \"maas-controller-66655566cf-nfw9v\" (UID: \"9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6\") " pod="opendatahub/maas-controller-66655566cf-nfw9v" Jul 16 09:23:50.155511 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:50.155382 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmrn2\" (UniqueName: \"kubernetes.io/projected/9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6-kube-api-access-rmrn2\") pod \"maas-controller-66655566cf-nfw9v\" (UID: \"9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6\") " pod="opendatahub/maas-controller-66655566cf-nfw9v" Jul 16 09:23:50.157950 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:50.157928 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6-webhook-cert\") pod \"maas-controller-66655566cf-nfw9v\" (UID: \"9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6\") " pod="opendatahub/maas-controller-66655566cf-nfw9v" Jul 16 09:23:50.164412 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:50.164386 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmrn2\" (UniqueName: \"kubernetes.io/projected/9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6-kube-api-access-rmrn2\") pod \"maas-controller-66655566cf-nfw9v\" (UID: \"9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6\") " pod="opendatahub/maas-controller-66655566cf-nfw9v" Jul 16 09:23:50.242623 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:50.242515 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66655566cf-nfw9v" Jul 16 09:23:50.381321 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:50.381297 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66655566cf-nfw9v"] Jul 16 09:23:50.384223 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:23:50.384193 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d1884be_cb9f_4c90_8b0f_a2af93b0b7d6.slice/crio-161cbece8d57d87facfa4a72f8d723ce0d39fd5c40d3efd35195de477654faea WatchSource:0}: Error finding container 161cbece8d57d87facfa4a72f8d723ce0d39fd5c40d3efd35195de477654faea: Status 404 returned error can't find the container with id 161cbece8d57d87facfa4a72f8d723ce0d39fd5c40d3efd35195de477654faea Jul 16 09:23:50.385488 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:50.385471 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 09:23:51.138243 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:51.138208 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66655566cf-nfw9v" event={"ID":"9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6","Type":"ContainerStarted","Data":"a67d4c5071e221a3bb3948b2b0c6800f0903762cc7f3c329b20fee12d270369a"} Jul 16 09:23:51.138481 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:51.138268 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66655566cf-nfw9v" event={"ID":"9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6","Type":"ContainerStarted","Data":"161cbece8d57d87facfa4a72f8d723ce0d39fd5c40d3efd35195de477654faea"} Jul 16 09:23:51.138481 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:51.138337 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66655566cf-nfw9v" Jul 16 09:23:51.158791 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:23:51.158733 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66655566cf-nfw9v" podStartSLOduration=1.601923386 podStartE2EDuration="2.158717593s" totalImagesPullingTime="556.794207ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:23:51.156323143 +0000 UTC m=+1106.565871429" watchObservedRunningTime="2026-07-16 09:23:51.158717593 +0000 UTC m=+1106.568265852" Jul 16 09:24:02.146403 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:24:02.146371 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66655566cf-nfw9v" Jul 16 09:25:25.176180 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:25:25.176151 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:25:25.178461 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:25:25.178438 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:29:41.311714 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.311674 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj"] Jul 16 09:29:41.315234 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.315216 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj" Jul 16 09:29:41.318149 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.318117 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z6zqn\"" Jul 16 09:29:41.318298 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.318117 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 09:29:41.318298 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.318117 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 09:29:41.326958 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.326928 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj"] Jul 16 09:29:41.458062 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.458017 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ef730e3-8ef2-4395-95ee-a6f6082767db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hjpnj\" (UID: \"5ef730e3-8ef2-4395-95ee-a6f6082767db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj" Jul 16 09:29:41.458279 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.458127 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvsfg\" (UniqueName: \"kubernetes.io/projected/5ef730e3-8ef2-4395-95ee-a6f6082767db-kube-api-access-hvsfg\") pod \"kuadrant-operator-controller-manager-86878c769f-hjpnj\" (UID: \"5ef730e3-8ef2-4395-95ee-a6f6082767db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj" Jul 16 09:29:41.559060 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.559013 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ef730e3-8ef2-4395-95ee-a6f6082767db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hjpnj\" (UID: \"5ef730e3-8ef2-4395-95ee-a6f6082767db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj" Jul 16 09:29:41.559263 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.559092 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvsfg\" (UniqueName: \"kubernetes.io/projected/5ef730e3-8ef2-4395-95ee-a6f6082767db-kube-api-access-hvsfg\") pod \"kuadrant-operator-controller-manager-86878c769f-hjpnj\" (UID: \"5ef730e3-8ef2-4395-95ee-a6f6082767db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj" Jul 16 09:29:41.559423 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.559404 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ef730e3-8ef2-4395-95ee-a6f6082767db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hjpnj\" (UID: \"5ef730e3-8ef2-4395-95ee-a6f6082767db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj" Jul 16 09:29:41.571026 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.570965 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvsfg\" (UniqueName: \"kubernetes.io/projected/5ef730e3-8ef2-4395-95ee-a6f6082767db-kube-api-access-hvsfg\") pod \"kuadrant-operator-controller-manager-86878c769f-hjpnj\" (UID: \"5ef730e3-8ef2-4395-95ee-a6f6082767db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj" Jul 16 09:29:41.628005 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.627966 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj" Jul 16 09:29:41.772047 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.772014 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj"] Jul 16 09:29:41.774038 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:29:41.774012 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef730e3_8ef2_4395_95ee_a6f6082767db.slice/crio-e241b994d622097d9e0bd25f8d77634ec9e5fafa8e04e7f72f5121d62b786812 WatchSource:0}: Error finding container e241b994d622097d9e0bd25f8d77634ec9e5fafa8e04e7f72f5121d62b786812: Status 404 returned error can't find the container with id e241b994d622097d9e0bd25f8d77634ec9e5fafa8e04e7f72f5121d62b786812 Jul 16 09:29:41.909728 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.909691 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj" event={"ID":"5ef730e3-8ef2-4395-95ee-a6f6082767db","Type":"ContainerStarted","Data":"422cd6c40bc9bae9e13005d06bf15758ec8b5dab2f82db19dab150bb20047dbc"} Jul 16 09:29:41.909961 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.909740 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj" event={"ID":"5ef730e3-8ef2-4395-95ee-a6f6082767db","Type":"ContainerStarted","Data":"e241b994d622097d9e0bd25f8d77634ec9e5fafa8e04e7f72f5121d62b786812"} Jul 16 09:29:41.909961 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.909853 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj" Jul 16 09:29:41.943687 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:41.943634 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj" podStartSLOduration=0.943616156 podStartE2EDuration="943.616156ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:29:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:29:41.940940469 +0000 UTC m=+1457.350488761" watchObservedRunningTime="2026-07-16 09:29:41.943616156 +0000 UTC m=+1457.353164474" Jul 16 09:29:52.917699 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:29:52.917609 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hjpnj" Jul 16 09:30:25.210849 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:30:25.210794 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:30:25.214007 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:30:25.213980 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:35:25.246862 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:35:25.246830 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:35:25.250350 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:35:25.250328 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:40:25.279329 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:40:25.279296 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:40:25.283354 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:40:25.283328 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:45:00.133393 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:00.133354 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736585-2pbn8"] Jul 16 09:45:00.137124 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:00.137105 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736585-2pbn8" Jul 16 09:45:00.140038 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:00.140012 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mtlcb\"" Jul 16 09:45:00.142196 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:00.142169 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8577c1-29736585-6k4ct"] Jul 16 09:45:00.145961 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:00.145936 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8577c1-29736585-6k4ct" Jul 16 09:45:00.146556 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:00.146533 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736585-2pbn8"] Jul 16 09:45:00.149386 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:00.149363 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736585-2pbn8" Jul 16 09:45:00.157112 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:00.157088 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8577c1-29736585-6k4ct"] Jul 16 09:45:00.160989 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:00.160967 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8577c1-29736585-6k4ct" Jul 16 09:45:00.311369 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:00.311342 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736585-2pbn8"] Jul 16 09:45:00.314179 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:45:00.314148 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef122f10_a5b7_42c1_a22f_83c62d2f3e4d.slice/crio-149ed63070d4f40fde88eed307ceb5c72aae1a2d196e30dbd5d7ec2cbf2d6143 WatchSource:0}: Error finding container 149ed63070d4f40fde88eed307ceb5c72aae1a2d196e30dbd5d7ec2cbf2d6143: Status 404 returned error can't find the container with id 149ed63070d4f40fde88eed307ceb5c72aae1a2d196e30dbd5d7ec2cbf2d6143 Jul 16 09:45:00.316020 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:00.315999 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 09:45:00.336320 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:00.336294 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8577c1-29736585-6k4ct"] Jul 16 09:45:00.338209 ip-10-0-134-16 kubenswrapper[2567]: W0716 09:45:00.338183 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e086468_bfd1_4dac_85a7_0f5452a17aba.slice/crio-03c91004fdf658b1b5c66706a3cb3483ed535c9b83ade95435a2da1e86292bd8 WatchSource:0}: Error finding container 03c91004fdf658b1b5c66706a3cb3483ed535c9b83ade95435a2da1e86292bd8: Status 404 returned error can't find the container with id 03c91004fdf658b1b5c66706a3cb3483ed535c9b83ade95435a2da1e86292bd8 Jul 16 09:45:00.666050 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:00.666011 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736585-2pbn8" event={"ID":"ef122f10-a5b7-42c1-a22f-83c62d2f3e4d","Type":"ContainerStarted","Data":"149ed63070d4f40fde88eed307ceb5c72aae1a2d196e30dbd5d7ec2cbf2d6143"} Jul 16 09:45:00.667293 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:00.667265 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8577c1-29736585-6k4ct" event={"ID":"9e086468-bfd1-4dac-85a7-0f5452a17aba","Type":"ContainerStarted","Data":"03c91004fdf658b1b5c66706a3cb3483ed535c9b83ade95435a2da1e86292bd8"} Jul 16 09:45:03.686395 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:03.686354 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8577c1-29736585-6k4ct" event={"ID":"9e086468-bfd1-4dac-85a7-0f5452a17aba","Type":"ContainerStarted","Data":"22063738e92ed5b487c81064bf44751c0490f8ff8e5870fc9e765a948264a4de"} Jul 16 09:45:03.687907 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:03.687883 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736585-2pbn8" event={"ID":"ef122f10-a5b7-42c1-a22f-83c62d2f3e4d","Type":"ContainerStarted","Data":"e36115dbda7ae84912eb8c41b3bede6b9e26ab657cd85c708761b8b1b58385d5"} Jul 16 09:45:03.716206 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:03.716156 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736585-2pbn8" podStartSLOduration=1.322290896 podStartE2EDuration="3.716139263s" totalImagesPullingTime="2.393848367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:45:03.715591132 +0000 UTC m=+2379.125139411" watchObservedRunningTime="2026-07-16 09:45:03.716139263 +0000 UTC m=+2379.125687522" Jul 16 09:45:03.716471 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:03.716451 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8577c1-29736585-6k4ct" podStartSLOduration=1.342286114 podStartE2EDuration="3.716442224s" totalImagesPullingTime="2.37415611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 09:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:45:03.701619391 +0000 UTC m=+2379.111167669" watchObservedRunningTime="2026-07-16 09:45:03.716442224 +0000 UTC m=+2379.125990483" Jul 16 09:45:04.695669 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:04.695628 2567 generic.go:356] "Generic (PLEG): container finished" podID="ef122f10-a5b7-42c1-a22f-83c62d2f3e4d" containerID="e36115dbda7ae84912eb8c41b3bede6b9e26ab657cd85c708761b8b1b58385d5" exitCode=0 Jul 16 09:45:04.696174 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:04.695707 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736585-2pbn8" event={"ID":"ef122f10-a5b7-42c1-a22f-83c62d2f3e4d","Type":"ContainerDied","Data":"e36115dbda7ae84912eb8c41b3bede6b9e26ab657cd85c708761b8b1b58385d5"} Jul 16 09:45:04.697475 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:04.697451 2567 generic.go:356] "Generic (PLEG): container finished" podID="9e086468-bfd1-4dac-85a7-0f5452a17aba" containerID="22063738e92ed5b487c81064bf44751c0490f8ff8e5870fc9e765a948264a4de" exitCode=0 Jul 16 09:45:04.697596 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:04.697507 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8577c1-29736585-6k4ct" event={"ID":"9e086468-bfd1-4dac-85a7-0f5452a17aba","Type":"ContainerDied","Data":"22063738e92ed5b487c81064bf44751c0490f8ff8e5870fc9e765a948264a4de"} Jul 16 09:45:05.874165 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:05.874130 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736585-2pbn8" Jul 16 09:45:05.877026 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:05.877004 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8577c1-29736585-6k4ct" Jul 16 09:45:06.709958 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:06.709917 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8577c1-29736585-6k4ct" event={"ID":"9e086468-bfd1-4dac-85a7-0f5452a17aba","Type":"ContainerDied","Data":"03c91004fdf658b1b5c66706a3cb3483ed535c9b83ade95435a2da1e86292bd8"} Jul 16 09:45:06.710143 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:06.709980 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03c91004fdf658b1b5c66706a3cb3483ed535c9b83ade95435a2da1e86292bd8" Jul 16 09:45:06.710143 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:06.709935 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8577c1-29736585-6k4ct" Jul 16 09:45:06.711204 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:06.711178 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736585-2pbn8" Jul 16 09:45:06.711347 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:06.711212 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736585-2pbn8" event={"ID":"ef122f10-a5b7-42c1-a22f-83c62d2f3e4d","Type":"ContainerDied","Data":"149ed63070d4f40fde88eed307ceb5c72aae1a2d196e30dbd5d7ec2cbf2d6143"} Jul 16 09:45:06.711347 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:06.711254 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="149ed63070d4f40fde88eed307ceb5c72aae1a2d196e30dbd5d7ec2cbf2d6143" Jul 16 09:45:25.316021 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:25.315890 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:45:25.342401 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:45:25.319086 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:49:37.412811 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:37.412771 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8577c1-29736585-6k4ct"] Jul 16 09:49:37.421131 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:37.421102 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8577c1-29736585-6k4ct"] Jul 16 09:49:39.324417 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:39.324366 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e086468-bfd1-4dac-85a7-0f5452a17aba" path="/var/lib/kubelet/pods/9e086468-bfd1-4dac-85a7-0f5452a17aba/volumes" Jul 16 09:49:50.256150 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:50.256111 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-66655566cf-nfw9v_9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6/manager/0.log" Jul 16 09:49:50.362060 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:50.362033 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9tw24_9a52779e-3df9-43cc-b885-6b04b6b1e9f0/server/0.log" Jul 16 09:49:50.599091 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:50.599060 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d64bd87c6-jdlvf_6f0706de-4ebf-4623-b150-50b95a7bc8ef/copy-manifests/0.log" Jul 16 09:49:50.606121 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:50.606094 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d64bd87c6-jdlvf_6f0706de-4ebf-4623-b150-50b95a7bc8ef/manager/0.log" Jul 16 09:49:51.220060 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:51.220023 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7fb9f65958-bb5t4_f2a4dd53-4139-4e35-a1fd-6e69388fd55c/maas-api/0.log" Jul 16 09:49:51.446092 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:51.446061 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29736585-2pbn8_ef122f10-a5b7-42c1-a22f-83c62d2f3e4d/cleanup/0.log" Jul 16 09:49:51.550395 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:51.550362 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-pfjkn_59ed8cf2-8dc1-43ed-8256-0cf2bb982be4/postgres/0.log" Jul 16 09:49:53.512521 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:53.512491 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hjpnj_5ef730e3-8ef2-4395-95ee-a6f6082767db/manager/0.log" Jul 16 09:49:54.241841 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:54.241787 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xd8df_c8e13b25-89a1-4473-9566-1d0f0aeab39b/discovery/0.log" Jul 16 09:49:54.725057 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:54.725025 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-jxj7r_9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd/payload-pre-processing/0.log" Jul 16 09:49:54.846119 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:54.846075 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-r6h5h_22460db3-fa52-4f5c-a034-44b273beeae9/payload-processing/0.log" Jul 16 09:49:55.332909 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:55.332877 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-8mlqm_a823d7f7-a285-4952-bb3b-48eada039386/main/0.log" Jul 16 09:49:55.444565 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:49:55.444528 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-dwlpl_1961b8e6-97be-444d-9aa4-127ee1126279/main/0.log" Jul 16 09:50:04.449340 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:04.449289 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lslxl_e84aa8e1-6582-487b-8261-61d858852378/global-pull-secret-syncer/0.log" Jul 16 09:50:04.634456 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:04.634426 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wvqmn_9eb0a970-f928-4d04-ab26-a2e41414db37/konnectivity-agent/0.log" Jul 16 09:50:04.683968 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:04.683933 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-16.ec2.internal_23e3fcaf29c3b1a8898aed672fb43ec0/haproxy/0.log" Jul 16 09:50:09.700707 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:09.700665 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hjpnj_5ef730e3-8ef2-4395-95ee-a6f6082767db/manager/0.log" Jul 16 09:50:11.275235 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:11.275206 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71/alertmanager/0.log" Jul 16 09:50:11.306865 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:11.306836 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71/config-reloader/0.log" Jul 16 09:50:11.338435 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:11.338404 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71/kube-rbac-proxy-web/0.log" Jul 16 09:50:11.373539 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:11.373500 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71/kube-rbac-proxy/0.log" Jul 16 09:50:11.397696 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:11.397658 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71/kube-rbac-proxy-metric/0.log" Jul 16 09:50:11.430614 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:11.430593 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71/prom-label-proxy/0.log" Jul 16 09:50:11.480790 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:11.480760 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_72abe2c8-0ca6-4c59-8cd5-45f1d7eb5a71/init-config-reloader/0.log" Jul 16 09:50:11.546510 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:11.546427 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-vbs25_320437bb-7863-4d89-8736-4c69fe673ad5/cluster-monitoring-operator/0.log" Jul 16 09:50:11.657014 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:11.656984 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6dfbf977df-nq9ln_a4816df6-0388-4b69-95d1-8f5e6fd3566b/metrics-server/0.log" Jul 16 09:50:11.689482 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:11.689456 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-2dx8n_8bf7f25f-65cb-4bf3-b422-b1a0ebbc339f/monitoring-plugin/0.log" Jul 16 09:50:11.725049 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:11.725019 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7996v_8b83a288-4615-4ee7-a2d4-fb0475223943/node-exporter/0.log" Jul 16 09:50:11.750353 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:11.750327 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7996v_8b83a288-4615-4ee7-a2d4-fb0475223943/kube-rbac-proxy/0.log" Jul 16 09:50:11.776455 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:11.776426 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7996v_8b83a288-4615-4ee7-a2d4-fb0475223943/init-textfile/0.log" Jul 16 09:50:12.124952 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:12.124921 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd854340-3c36-4687-bdbc-aa05194036ee/prometheus/0.log" Jul 16 09:50:12.155243 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:12.155213 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd854340-3c36-4687-bdbc-aa05194036ee/config-reloader/0.log" Jul 16 09:50:12.183765 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:12.183736 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd854340-3c36-4687-bdbc-aa05194036ee/thanos-sidecar/0.log" Jul 16 09:50:12.209632 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:12.209607 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd854340-3c36-4687-bdbc-aa05194036ee/kube-rbac-proxy-web/0.log" Jul 16 09:50:12.242990 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:12.242965 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd854340-3c36-4687-bdbc-aa05194036ee/kube-rbac-proxy/0.log" Jul 16 09:50:12.267933 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:12.267901 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd854340-3c36-4687-bdbc-aa05194036ee/kube-rbac-proxy-thanos/0.log" Jul 16 09:50:12.294190 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:12.294144 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dd854340-3c36-4687-bdbc-aa05194036ee/init-config-reloader/0.log" Jul 16 09:50:12.499349 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:12.499266 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fc66995cd-gwscm_f3db39f0-a2c4-4cf3-830a-0fa307cba303/thanos-query/0.log" Jul 16 09:50:12.523603 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:12.523562 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fc66995cd-gwscm_f3db39f0-a2c4-4cf3-830a-0fa307cba303/kube-rbac-proxy-web/0.log" Jul 16 09:50:12.545113 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:12.545081 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fc66995cd-gwscm_f3db39f0-a2c4-4cf3-830a-0fa307cba303/kube-rbac-proxy/0.log" Jul 16 09:50:12.572626 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:12.572595 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fc66995cd-gwscm_f3db39f0-a2c4-4cf3-830a-0fa307cba303/prom-label-proxy/0.log" Jul 16 09:50:12.598026 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:12.597992 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fc66995cd-gwscm_f3db39f0-a2c4-4cf3-830a-0fa307cba303/kube-rbac-proxy-rules/0.log" Jul 16 09:50:12.619671 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:12.619639 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fc66995cd-gwscm_f3db39f0-a2c4-4cf3-830a-0fa307cba303/kube-rbac-proxy-metrics/0.log" Jul 16 09:50:13.721346 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.721304 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm"] Jul 16 09:50:13.721961 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.721942 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef122f10-a5b7-42c1-a22f-83c62d2f3e4d" containerName="cleanup" Jul 16 09:50:13.722059 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.721963 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef122f10-a5b7-42c1-a22f-83c62d2f3e4d" containerName="cleanup" Jul 16 09:50:13.722059 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.721974 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e086468-bfd1-4dac-85a7-0f5452a17aba" containerName="cleanup" Jul 16 09:50:13.722059 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.721984 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e086468-bfd1-4dac-85a7-0f5452a17aba" containerName="cleanup" Jul 16 09:50:13.722216 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.722125 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e086468-bfd1-4dac-85a7-0f5452a17aba" containerName="cleanup" Jul 16 09:50:13.722216 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.722142 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef122f10-a5b7-42c1-a22f-83c62d2f3e4d" containerName="cleanup" Jul 16 09:50:13.725458 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.725437 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.728781 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.728761 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xc7d7\"/\"openshift-service-ca.crt\"" Jul 16 09:50:13.729805 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.729782 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xc7d7\"/\"kube-root-ca.crt\"" Jul 16 09:50:13.729920 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.729784 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xc7d7\"/\"default-dockercfg-54m6m\"" Jul 16 09:50:13.737196 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.737169 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm"] Jul 16 09:50:13.754527 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.754484 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-xqqv6_7d93ba98-9730-4d8e-9ab5-d5111a98387c/networking-console-plugin/0.log" Jul 16 09:50:13.831707 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.831669 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d2d354c-0a79-4726-ba4d-d22f688cc215-sys\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.831955 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.831712 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dglk\" (UniqueName: \"kubernetes.io/projected/5d2d354c-0a79-4726-ba4d-d22f688cc215-kube-api-access-6dglk\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.831955 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.831742 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d2d354c-0a79-4726-ba4d-d22f688cc215-podres\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.831955 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.831873 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d2d354c-0a79-4726-ba4d-d22f688cc215-proc\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.832093 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.831972 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d2d354c-0a79-4726-ba4d-d22f688cc215-lib-modules\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.933450 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.933402 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d2d354c-0a79-4726-ba4d-d22f688cc215-sys\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.933660 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.933456 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dglk\" (UniqueName: \"kubernetes.io/projected/5d2d354c-0a79-4726-ba4d-d22f688cc215-kube-api-access-6dglk\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.933660 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.933490 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d2d354c-0a79-4726-ba4d-d22f688cc215-podres\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.933660 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.933537 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d2d354c-0a79-4726-ba4d-d22f688cc215-sys\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.933660 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.933547 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d2d354c-0a79-4726-ba4d-d22f688cc215-proc\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.933660 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.933629 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d2d354c-0a79-4726-ba4d-d22f688cc215-lib-modules\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.933660 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.933648 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d2d354c-0a79-4726-ba4d-d22f688cc215-proc\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.933942 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.933649 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d2d354c-0a79-4726-ba4d-d22f688cc215-podres\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.933942 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.933746 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d2d354c-0a79-4726-ba4d-d22f688cc215-lib-modules\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:13.942280 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:13.942253 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dglk\" (UniqueName: \"kubernetes.io/projected/5d2d354c-0a79-4726-ba4d-d22f688cc215-kube-api-access-6dglk\") pod \"perf-node-gather-daemonset-c5wgm\" (UID: \"5d2d354c-0a79-4726-ba4d-d22f688cc215\") " pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:14.037464 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:14.037435 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:14.177916 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:14.177888 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm"] Jul 16 09:50:14.309713 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:14.309682 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" event={"ID":"5d2d354c-0a79-4726-ba4d-d22f688cc215","Type":"ContainerStarted","Data":"70203902c2335df1e24e626eeea36669b17e4c834162b513f3ee81574054bcbd"} Jul 16 09:50:14.309876 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:14.309738 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:14.328269 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:14.328216 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" podStartSLOduration=1.328196046 podStartE2EDuration="1.328196046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 09:50:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 09:50:14.325889141 +0000 UTC m=+2689.735437436" watchObservedRunningTime="2026-07-16 09:50:14.328196046 +0000 UTC m=+2689.737744307" Jul 16 09:50:14.409426 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:14.409399 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:50:14.417764 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:14.417736 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/2.log" Jul 16 09:50:15.316899 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:15.316851 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" event={"ID":"5d2d354c-0a79-4726-ba4d-d22f688cc215","Type":"ContainerStarted","Data":"09d679d8d39505d79de6f10c7d707a93bd7a1446cfb90a0d97ed0510145962f7"} Jul 16 09:50:15.462597 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:15.462562 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-zfjdj_8322ecd6-180e-462a-97d2-8cd3942f208d/volume-data-source-validator/0.log" Jul 16 09:50:16.534758 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:16.534673 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-szwcd_f1ef4d9f-e004-434d-af44-a51c96bfeb32/dns/0.log" Jul 16 09:50:16.556072 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:16.556042 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-szwcd_f1ef4d9f-e004-434d-af44-a51c96bfeb32/kube-rbac-proxy/0.log" Jul 16 09:50:16.635010 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:16.634983 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-78b27_850750a5-782b-490b-b29a-ae2e05d1aff8/dns-node-resolver/0.log" Jul 16 09:50:17.302128 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:17.302092 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mprh4_3f14599c-3b35-405b-a145-1a861e050d1c/node-ca/0.log" Jul 16 09:50:18.415089 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:18.415057 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xd8df_c8e13b25-89a1-4473-9566-1d0f0aeab39b/discovery/0.log" Jul 16 09:50:18.573431 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:18.573399 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-jxj7r_9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd/payload-pre-processing/0.log" Jul 16 09:50:18.614579 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:18.614549 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-r6h5h_22460db3-fa52-4f5c-a034-44b273beeae9/payload-processing/0.log" Jul 16 09:50:19.301641 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:19.301602 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kzpnm_d2d206a0-1603-4fd8-936a-dacba9979e0c/serve-healthcheck-canary/0.log" Jul 16 09:50:19.878117 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:19.878085 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-5gbht_9fa2a7c4-5499-408f-9c98-acc1debc7be0/insights-operator/0.log" Jul 16 09:50:19.878859 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:19.878836 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-5gbht_9fa2a7c4-5499-408f-9c98-acc1debc7be0/insights-operator/1.log" Jul 16 09:50:20.017994 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:20.017965 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mm9f5_4323d88c-6520-4845-816b-aeb12c4afc85/kube-rbac-proxy/0.log" Jul 16 09:50:20.044837 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:20.044799 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mm9f5_4323d88c-6520-4845-816b-aeb12c4afc85/exporter/0.log" Jul 16 09:50:20.073520 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:20.073487 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mm9f5_4323d88c-6520-4845-816b-aeb12c4afc85/extractor/0.log" Jul 16 09:50:21.333143 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:21.333117 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xc7d7/perf-node-gather-daemonset-c5wgm" Jul 16 09:50:22.644682 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:22.644646 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-66655566cf-nfw9v_9d1884be-cb9f-4c90-8b0f-a2af93b0b7d6/manager/0.log" Jul 16 09:50:22.675927 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:22.675897 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9tw24_9a52779e-3df9-43cc-b885-6b04b6b1e9f0/server/0.log" Jul 16 09:50:22.779292 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:22.779260 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d64bd87c6-jdlvf_6f0706de-4ebf-4623-b150-50b95a7bc8ef/manager/0.log" Jul 16 09:50:22.806449 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:22.806421 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d64bd87c6-jdlvf_6f0706de-4ebf-4623-b150-50b95a7bc8ef/copy-manifests/0.log" Jul 16 09:50:23.558343 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:23.558308 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xd8df_c8e13b25-89a1-4473-9566-1d0f0aeab39b/discovery/0.log" Jul 16 09:50:23.854730 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:23.854698 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-jxj7r_9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd/payload-pre-processing/0.log" Jul 16 09:50:23.909679 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:23.909644 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-r6h5h_22460db3-fa52-4f5c-a034-44b273beeae9/payload-processing/0.log" Jul 16 09:50:24.352327 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:24.352296 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6ddb4bcd65-ktfzb_7dc06d9a-8a07-4ca6-b2b4-b7b133e68c40/manager/0.log" Jul 16 09:50:25.358462 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:25.358347 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:50:25.375580 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:25.364851 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-f4qh9_42c8464a-45ad-406d-b06a-cc2e8c6d1cbc/console-operator/1.log" Jul 16 09:50:25.447709 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:25.447667 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xd8df_c8e13b25-89a1-4473-9566-1d0f0aeab39b/discovery/0.log" Jul 16 09:50:25.590203 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:25.590166 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-jxj7r_9f67aea2-02a5-45eb-bb5a-17c92d0bf8fd/payload-pre-processing/0.log" Jul 16 09:50:25.628551 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:25.628466 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-r6h5h_22460db3-fa52-4f5c-a034-44b273beeae9/payload-processing/0.log" Jul 16 09:50:29.575959 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:29.575913 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-94bkn_0aaa2820-a527-477c-a6f0-e95b97f14f5a/migrator/0.log" Jul 16 09:50:29.600740 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:29.600692 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-94bkn_0aaa2820-a527-477c-a6f0-e95b97f14f5a/graceful-termination/0.log" Jul 16 09:50:31.431603 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:31.431571 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sd79d_21c0ee12-ae01-43a6-83b1-af6b69997b76/kube-multus-additional-cni-plugins/0.log" Jul 16 09:50:31.456474 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:31.456449 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sd79d_21c0ee12-ae01-43a6-83b1-af6b69997b76/egress-router-binary-copy/0.log" Jul 16 09:50:31.479126 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:31.479095 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sd79d_21c0ee12-ae01-43a6-83b1-af6b69997b76/cni-plugins/0.log" Jul 16 09:50:31.506490 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:31.506458 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sd79d_21c0ee12-ae01-43a6-83b1-af6b69997b76/bond-cni-plugin/0.log" Jul 16 09:50:31.535405 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:31.535379 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sd79d_21c0ee12-ae01-43a6-83b1-af6b69997b76/routeoverride-cni/0.log" Jul 16 09:50:31.560498 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:31.560470 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sd79d_21c0ee12-ae01-43a6-83b1-af6b69997b76/whereabouts-cni-bincopy/0.log" Jul 16 09:50:31.587608 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:31.587576 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sd79d_21c0ee12-ae01-43a6-83b1-af6b69997b76/whereabouts-cni/0.log" Jul 16 09:50:31.676695 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:31.676650 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-v6mrg_6c4d81b5-6013-424d-be5e-ec4877e5d16c/kube-multus/0.log" Jul 16 09:50:31.754245 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:31.754157 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ph7bq_9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd/network-metrics-daemon/0.log" Jul 16 09:50:31.777379 ip-10-0-134-16 kubenswrapper[2567]: I0716 09:50:31.777348 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ph7bq_9f9f96c9-54b3-47a9-9329-6ac7e8ded7bd/kube-rbac-proxy/0.log"