Jul 06 19:25:35.412950 ip-10-0-141-33 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 19:25:35.412964 ip-10-0-141-33 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 19:25:35.412973 ip-10-0-141-33 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 19:25:35.413290 ip-10-0-141-33 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 19:25:45.502344 ip-10-0-141-33 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 19:25:45.502362 ip-10-0-141-33 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 6f8b338c5a914cc08c105266c4af98ba -- Jul 06 19:27:45.016626 ip-10-0-141-33 systemd[1]: Starting Kubernetes Kubelet... Jul 06 19:27:45.411436 ip-10-0-141-33 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:27:45.411436 ip-10-0-141-33 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 19:27:45.411436 ip-10-0-141-33 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:27:45.411436 ip-10-0-141-33 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 19:27:45.411436 ip-10-0-141-33 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:27:45.412204 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.412114 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 19:27:45.415681 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415663 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:27:45.415681 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415680 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:27:45.415681 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415684 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415688 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415691 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415696 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415700 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415704 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415707 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415710 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415714 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415717 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415719 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415723 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415726 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415734 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415737 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415739 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415742 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415744 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415747 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:27:45.415784 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415749 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415752 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415755 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415757 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415760 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415762 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415765 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415767 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415770 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415773 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415775 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415777 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415780 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415783 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415786 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415788 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415791 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415794 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415797 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415801 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:27:45.416227 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415804 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415807 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415809 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415812 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415814 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415817 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415819 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415821 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415824 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415826 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415828 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415831 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415833 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415836 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415839 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415842 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415845 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415848 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415850 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:27:45.416733 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415854 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415858 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415860 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415863 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415865 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415868 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415871 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415873 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415876 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415880 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415884 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415888 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415891 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415893 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415896 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415900 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415902 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415905 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415907 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415910 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:27:45.417191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415912 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415915 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415918 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415920 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415923 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415925 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415927 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415931 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415934 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415937 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415939 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415942 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415945 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415947 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415950 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415952 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415955 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415958 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.415960 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416693 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:27:45.417680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416700 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416704 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416709 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416712 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416715 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416718 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416720 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416723 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416726 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416729 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416731 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416734 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416737 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416739 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416742 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416744 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416747 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416749 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416752 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416755 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:27:45.418155 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416758 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416761 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416763 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416766 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416769 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416771 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416774 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416776 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416779 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416781 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416784 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416786 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416789 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416791 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416794 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416796 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416798 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416802 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416804 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416807 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:27:45.418666 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416810 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416812 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416815 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416817 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416819 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416822 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416825 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416827 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416830 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416832 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416835 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416839 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416842 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416844 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416846 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416849 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416851 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416854 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416856 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:27:45.419143 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416859 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416861 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416864 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416866 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416871 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416874 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416877 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416880 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416883 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416886 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416888 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416892 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416894 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416897 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416899 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416902 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416905 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416909 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416912 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:27:45.419618 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416915 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416917 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416920 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416922 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416925 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416928 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416932 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416934 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416937 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416939 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416942 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416945 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416947 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416950 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416953 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416955 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416958 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416960 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416963 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.416965 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417359 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:27:45.420071 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417364 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417366 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417369 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417373 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417376 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417379 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417382 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417385 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417387 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417390 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417393 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417397 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417400 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417403 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417406 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417408 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417411 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417414 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417417 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417419 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:27:45.420612 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417423 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417427 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417432 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417435 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417437 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417440 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417442 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417445 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417448 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417450 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417453 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417455 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417458 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417460 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417463 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417468 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417471 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417474 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417476 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417479 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:27:45.421099 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417481 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417484 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417486 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417489 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417491 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417494 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417496 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417498 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417501 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417504 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417506 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417509 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417511 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417514 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417516 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417519 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417521 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417524 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417527 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417530 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:27:45.421592 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417532 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417535 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417537 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417539 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417542 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417544 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417547 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417550 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417553 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417556 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417559 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417561 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417565 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417568 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417570 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417573 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417575 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417578 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417580 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417583 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:27:45.422073 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417585 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417589 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417592 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417594 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417597 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417599 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417602 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417605 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417607 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417610 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417612 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417615 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417618 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417620 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417623 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417625 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417627 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.417630 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417704 2561 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417719 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 19:27:45.422624 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417725 2561 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417730 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417734 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417737 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417742 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417747 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417750 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417753 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417756 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417759 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417762 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417765 2561 flags.go:64] FLAG: --cgroup-root="" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417769 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417772 2561 flags.go:64] FLAG: --client-ca-file="" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417774 2561 flags.go:64] FLAG: --cloud-provider="external" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417777 2561 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417787 2561 flags.go:64] FLAG: --cluster-domain="" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417790 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417793 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417796 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417799 2561 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417803 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417806 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 19:27:45.423103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417809 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417813 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417816 2561 flags.go:64] FLAG: --contention-profiling="false" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417818 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417821 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417825 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417827 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417831 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417835 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417838 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417841 2561 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417844 2561 flags.go:64] FLAG: --enable-server="true" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417847 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417852 2561 flags.go:64] FLAG: --event-burst="100" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417855 2561 flags.go:64] FLAG: --event-qps="50" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417859 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417861 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417864 2561 flags.go:64] FLAG: --eviction-hard="" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417869 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417871 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417874 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417878 2561 flags.go:64] FLAG: --eviction-soft="" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417880 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417883 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417886 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 19:27:45.423656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417889 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417892 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417895 2561 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417898 2561 flags.go:64] FLAG: --feature-gates="" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417902 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417905 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417908 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417911 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417914 2561 flags.go:64] FLAG: --healthz-port="10248" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417917 2561 flags.go:64] FLAG: --help="false" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417920 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-141-33.ec2.internal" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417923 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417926 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417929 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417932 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417936 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417939 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417942 2561 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417944 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417947 2561 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417951 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417954 2561 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417957 2561 flags.go:64] FLAG: --kube-reserved="" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417961 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 19:27:45.424336 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417963 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417966 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417969 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417972 2561 flags.go:64] FLAG: --lock-file="" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417975 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417978 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417981 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417986 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417989 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417992 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417995 2561 flags.go:64] FLAG: --logging-format="text" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.417998 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418001 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418004 2561 flags.go:64] FLAG: --manifest-url="" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418007 2561 flags.go:64] FLAG: --manifest-url-header="" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418012 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418015 2561 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418019 2561 flags.go:64] FLAG: --max-pods="110" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418021 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418025 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418027 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418030 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418033 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418036 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418039 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 19:27:45.424977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418046 2561 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418049 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418052 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418055 2561 flags.go:64] FLAG: --pod-cidr="" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418058 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418064 2561 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418067 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418070 2561 flags.go:64] FLAG: --pods-per-core="0" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418072 2561 flags.go:64] FLAG: --port="10250" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418075 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418078 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-057720f398a5fe42f" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418081 2561 flags.go:64] FLAG: --qos-reserved="" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418084 2561 flags.go:64] FLAG: --read-only-port="10255" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418087 2561 flags.go:64] FLAG: --register-node="true" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418090 2561 flags.go:64] FLAG: --register-with-taints="" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418093 2561 flags.go:64] FLAG: --registry-burst="10" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418096 2561 flags.go:64] FLAG: --registry-qps="5" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418099 2561 flags.go:64] FLAG: --reserved-cpus="" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418101 2561 flags.go:64] FLAG: --reserved-memory="" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418105 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418108 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418113 2561 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418115 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418118 2561 flags.go:64] FLAG: --runonce="false" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418121 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 19:27:45.425626 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418124 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418127 2561 flags.go:64] FLAG: --seccomp-default="false" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418129 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418133 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418136 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418139 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418143 2561 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418146 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418149 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418152 2561 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418154 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418157 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418161 2561 flags.go:64] FLAG: --system-cgroups="" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418164 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418170 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418172 2561 flags.go:64] FLAG: --tls-cert-file="" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418175 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418179 2561 flags.go:64] FLAG: --tls-min-version="" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418182 2561 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418185 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418188 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418191 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418194 2561 flags.go:64] FLAG: --v="2" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418198 2561 flags.go:64] FLAG: --version="false" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418203 2561 flags.go:64] FLAG: --vmodule="" Jul 06 19:27:45.426235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418206 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.418210 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418334 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418341 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418344 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418347 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418350 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418352 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418355 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418357 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418360 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418362 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418365 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418369 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418371 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418374 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418377 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418379 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418382 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418385 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:27:45.426861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418388 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418391 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418393 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418396 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418398 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418401 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418403 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418407 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418411 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418414 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418417 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418419 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418422 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418425 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418428 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418432 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418435 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418438 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418440 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418443 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:27:45.427673 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418446 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418448 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418450 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418453 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418455 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418460 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418462 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418466 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418469 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418471 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418474 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418476 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418479 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418482 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418484 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418487 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418489 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418492 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418494 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418497 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:27:45.428350 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418500 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418502 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418505 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418508 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418512 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418515 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418518 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418522 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418526 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418528 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418531 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418534 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418537 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418540 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418542 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418545 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418547 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418551 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418554 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:27:45.428861 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418557 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418560 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418563 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.418565 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419403 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419409 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419413 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419418 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419421 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419424 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419427 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419429 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419432 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419435 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419437 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419440 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419443 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419446 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419448 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419451 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:27:45.429338 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419455 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:27:45.429840 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.419458 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:27:45.429840 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.419462 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:27:45.429840 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.426001 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 19:27:45.429840 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.426017 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 19:27:45.429840 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.426035 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 19:27:45.429840 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.426041 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 19:27:45.429840 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426099 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:27:45.429840 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426104 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:27:45.429840 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426108 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:27:45.429840 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426113 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:27:45.429840 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426117 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:27:45.429840 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426121 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:27:45.429840 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426124 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:27:45.429840 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426127 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426130 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426132 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426135 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426138 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426141 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426144 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426146 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426150 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426154 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426157 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426160 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426163 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426165 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426168 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426171 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426174 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426177 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426179 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426182 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:27:45.430243 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426184 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426187 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426191 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426194 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426197 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426199 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426202 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426204 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426207 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426210 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426212 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426214 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426217 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426219 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426222 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426224 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426227 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426229 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426232 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426234 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:27:45.430852 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426237 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426240 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426243 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426246 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426249 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426252 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426254 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426257 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426259 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426262 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426265 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426268 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426271 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426273 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426277 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426280 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426282 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426285 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426299 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:27:45.431355 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426302 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426305 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426308 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426312 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426316 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426319 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426321 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426324 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426326 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426329 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426332 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426334 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426337 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426339 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426342 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426345 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426347 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426350 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426352 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:27:45.431817 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426355 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426358 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426360 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426363 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426366 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426369 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426371 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426374 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426378 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426380 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426384 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426387 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426389 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426391 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.426396 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:27:45.432281 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426512 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426517 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426520 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426523 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426526 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426529 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426532 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426535 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426538 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426541 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426544 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426547 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426549 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426552 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426555 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426558 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426560 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426563 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426565 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426568 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:27:45.432680 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426571 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426574 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426577 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426580 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426583 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426585 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426588 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426592 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426597 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426599 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426602 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426604 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426607 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426609 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426612 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426615 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426617 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426621 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426624 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:27:45.433164 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426627 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426630 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426633 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426636 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426639 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426642 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426644 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426648 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426650 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426653 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426655 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426658 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426660 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426663 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426665 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426667 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426670 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426673 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426676 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426678 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:27:45.433630 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426681 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426683 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426686 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426689 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426693 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426696 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426699 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426701 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426704 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426707 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426709 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426712 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426714 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426717 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426719 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426722 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426724 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426727 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426730 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:27:45.434107 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426732 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426735 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426737 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426740 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426743 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426745 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426747 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426750 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426752 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426755 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426758 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426761 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426763 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426766 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426768 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426771 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426774 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426777 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426779 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:27:45.434571 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426782 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:27:45.435017 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:45.426784 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:27:45.435017 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.426789 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:27:45.435017 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.429736 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 19:27:45.435017 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.431809 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 19:27:45.435017 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.433233 2561 server.go:1019] "Starting client certificate rotation" Jul 06 19:27:45.435017 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.433325 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:27:45.435017 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.433365 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:27:45.435928 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.435913 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:27:45.437247 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.437226 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:27:45.447855 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.447829 2561 log.go:25] "Validated CRI v1 runtime API" Jul 06 19:27:45.450069 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.450041 2561 log.go:25] "Validated CRI v1 image API" Jul 06 19:27:45.451049 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.451028 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 19:27:45.455867 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.455837 2561 fs.go:135] Filesystem UUIDs: map[049dec18-673a-48b1-95a8-ee4308968061:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ae2ebdf4-4e4b-40ba-8dc4-f38175928de7:/dev/nvme0n1p4] Jul 06 19:27:45.455950 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.455869 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 19:27:45.462879 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.462752 2561 manager.go:216] Machine: {Timestamp:2026-07-06 19:27:45.461291673 +0000 UTC m=+0.328967229 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098906 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e7f7da96917b80ac6e471ef280c45 SystemUUID:ec2e7f7d-a969-17b8-0ac6-e471ef280c45 BootID:6f8b338c-5a91-4cc0-8c10-5266c4af98ba Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c8:b6:61:a2:bd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c8:b6:61:a2:bd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:4d:b2:ec:b3:53 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 19:27:45.462879 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.462872 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 19:27:45.463010 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.462975 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 19:27:45.466427 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.466394 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 19:27:45.466600 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.466430 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-33.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 19:27:45.467219 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.467208 2561 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 19:27:45.467255 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.467223 2561 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 19:27:45.467255 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.467231 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:27:45.468404 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.468391 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:27:45.469637 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.469623 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 19:27:45.472759 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.472746 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:27:45.472892 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.472882 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 19:27:45.473627 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.473617 2561 kubelet.go:486] "Attempting to sync node with API server" Jul 06 19:27:45.473665 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.473633 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 19:27:45.473665 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.473650 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 19:27:45.473665 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.473662 2561 kubelet.go:398] "Adding apiserver pod source" Jul 06 19:27:45.473749 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.473671 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 19:27:45.474069 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.474057 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:27:45.474104 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.474077 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:27:45.481682 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.481658 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 19:27:45.483619 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.483605 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 19:27:45.483667 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.483638 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 19:27:45.484721 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.484710 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 19:27:45.484781 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.484727 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 19:27:45.484781 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.484734 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 19:27:45.484781 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.484741 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 19:27:45.484781 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.484750 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 19:27:45.484781 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.484757 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 19:27:45.484781 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.484766 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 19:27:45.484781 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.484772 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 19:27:45.484781 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.484780 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 19:27:45.484781 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.484786 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 19:27:45.485023 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.484795 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 19:27:45.485023 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.484801 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 19:27:45.486156 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.486138 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:27:45.486281 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.486269 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 19:27:45.486318 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.486283 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 19:27:45.490071 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.490057 2561 server.go:1268] "Started kubelet" Jul 06 19:27:45.490266 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.490163 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 19:27:45.490343 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.490239 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 19:27:45.490343 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.490312 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 19:27:45.491221 ip-10-0-141-33 systemd[1]: Started Kubernetes Kubelet. Jul 06 19:27:45.493065 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.492799 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 19:27:45.493947 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.493922 2561 server.go:310] "Adding debug handlers to kubelet server" Jul 06 19:27:45.497427 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.497408 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 19:27:45.498789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.498773 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 19:27:45.498789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.498785 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 19:27:45.498941 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.498908 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:45.499378 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.499357 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 19:27:45.499378 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.499380 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 19:27:45.499526 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.499358 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 19:27:45.499526 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.499498 2561 reconstruct.go:97] "Volume reconstruction finished" Jul 06 19:27:45.499526 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.499508 2561 reconciler.go:29] "Reconciler: start to sync state" Jul 06 19:27:45.499659 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.499641 2561 factory.go:55] Registering systemd factory Jul 06 19:27:45.499659 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.499656 2561 factory.go:223] Registration of the systemd container factory successfully Jul 06 19:27:45.499893 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.499879 2561 factory.go:153] Registering CRI-O factory Jul 06 19:27:45.499893 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.499894 2561 factory.go:223] Registration of the crio container factory successfully Jul 06 19:27:45.500027 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.499971 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 19:27:45.500027 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.499998 2561 factory.go:103] Registering Raw factory Jul 06 19:27:45.500027 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.500019 2561 manager.go:1196] Started watching for new ooms in manager Jul 06 19:27:45.500434 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.500419 2561 manager.go:349] Starting recovery of all containers Jul 06 19:27:45.500786 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.500766 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-33.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 19:27:45.500885 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.500861 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-33.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 19:27:45.500953 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.500935 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 19:27:45.502568 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.501064 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-33.ec2.internal.18bfc9fce604e023 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-33.ec2.internal,UID:ip-10-0-141-33.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-33.ec2.internal,},FirstTimestamp:2026-07-06 19:27:45.490026531 +0000 UTC m=+0.357702080,LastTimestamp:2026-07-06 19:27:45.490026531 +0000 UTC m=+0.357702080,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-33.ec2.internal,}" Jul 06 19:27:45.514665 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.514648 2561 manager.go:354] Recovery completed Jul 06 19:27:45.519728 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.519711 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:27:45.522290 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.522266 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:27:45.522357 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.522304 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:27:45.522357 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.522315 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:27:45.522918 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.522895 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 19:27:45.522918 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.522909 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 19:27:45.523053 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.522933 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:27:45.523053 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.522935 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:27:45.523549 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.523526 2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-33.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 19:27:45.523549 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.523530 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 19:27:45.523754 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.523740 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:27:45.523792 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.523763 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:27:45.523792 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.523778 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:27:45.524608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.524591 2561 policy_none.go:49] "None policy: Start" Jul 06 19:27:45.524608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.524609 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 19:27:45.524701 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.524622 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 19:27:45.525883 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.525872 2561 policy_none.go:47] "Start" Jul 06 19:27:45.542544 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.542456 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-33.ec2.internal.18bfc9fce7f13072 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-33.ec2.internal,UID:ip-10-0-141-33.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-33.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-33.ec2.internal,},FirstTimestamp:2026-07-06 19:27:45.522290802 +0000 UTC m=+0.389966351,LastTimestamp:2026-07-06 19:27:45.522290802 +0000 UTC m=+0.389966351,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-33.ec2.internal,}" Jul 06 19:27:45.549599 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.549517 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-33.ec2.internal.18bfc9fce7f17b36 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-33.ec2.internal,UID:ip-10-0-141-33.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-33.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-33.ec2.internal,},FirstTimestamp:2026-07-06 19:27:45.522309942 +0000 UTC m=+0.389985492,LastTimestamp:2026-07-06 19:27:45.522309942 +0000 UTC m=+0.389985492,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-33.ec2.internal,}" Jul 06 19:27:45.555630 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.555557 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-33.ec2.internal.18bfc9fce7f1a034 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-33.ec2.internal,UID:ip-10-0-141-33.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-33.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-33.ec2.internal,},FirstTimestamp:2026-07-06 19:27:45.522319412 +0000 UTC m=+0.389994962,LastTimestamp:2026-07-06 19:27:45.522319412 +0000 UTC m=+0.389994962,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-33.ec2.internal,}" Jul 06 19:27:45.563440 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.563341 2561 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-33.ec2.internal.18bfc9fce7f13072\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-33.ec2.internal.18bfc9fce7f13072 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-33.ec2.internal,UID:ip-10-0-141-33.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-33.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-33.ec2.internal,},FirstTimestamp:2026-07-06 19:27:45.522290802 +0000 UTC m=+0.389966351,LastTimestamp:2026-07-06 19:27:45.523753467 +0000 UTC m=+0.391429017,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-33.ec2.internal,}" Jul 06 19:27:45.570227 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.570205 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zwrl4" Jul 06 19:27:45.570415 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.570304 2561 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-33.ec2.internal.18bfc9fce7f17b36\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-33.ec2.internal.18bfc9fce7f17b36 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-33.ec2.internal,UID:ip-10-0-141-33.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-33.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-33.ec2.internal,},FirstTimestamp:2026-07-06 19:27:45.522309942 +0000 UTC m=+0.389985492,LastTimestamp:2026-07-06 19:27:45.523770111 +0000 UTC m=+0.391445662,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-33.ec2.internal,}" Jul 06 19:27:45.580607 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.580506 2561 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-33.ec2.internal.18bfc9fce7f1a034\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-33.ec2.internal.18bfc9fce7f1a034 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-33.ec2.internal,UID:ip-10-0-141-33.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-33.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-33.ec2.internal,},FirstTimestamp:2026-07-06 19:27:45.522319412 +0000 UTC m=+0.389994962,LastTimestamp:2026-07-06 19:27:45.523784419 +0000 UTC m=+0.391459974,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-33.ec2.internal,}" Jul 06 19:27:45.582541 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.582526 2561 manager.go:337] "Starting Device Plugin manager" Jul 06 19:27:45.582609 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.582569 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 19:27:45.582609 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.582580 2561 server.go:87] "Starting device plugin registration server" Jul 06 19:27:45.582908 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.582886 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 19:27:45.583020 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.582903 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 19:27:45.583078 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.583027 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 19:27:45.583126 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.583115 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 19:27:45.583126 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.583125 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 19:27:45.583750 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.583727 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 19:27:45.583830 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.583791 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:45.591766 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.591742 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zwrl4" Jul 06 19:27:45.679047 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.678949 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 19:27:45.680332 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.680291 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 19:27:45.680448 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.680342 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 19:27:45.680448 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.680388 2561 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 19:27:45.680448 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.680434 2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 19:27:45.683079 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.683057 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:27:45.683535 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.683515 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:27:45.683933 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.683916 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:27:45.684020 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.683947 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:27:45.684020 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.683960 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:27:45.684020 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.683992 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-33.ec2.internal" Jul 06 19:27:45.707416 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.707381 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-33.ec2.internal" Jul 06 19:27:45.707416 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.707414 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-33.ec2.internal\": node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:45.761916 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.761882 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:45.780932 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.780896 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal"] Jul 06 19:27:45.781001 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.780996 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:27:45.782028 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.782010 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:27:45.782085 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.782043 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:27:45.782085 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.782060 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:27:45.783499 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.783484 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:27:45.783611 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.783597 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Jul 06 19:27:45.783657 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.783636 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:27:45.784222 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.784204 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:27:45.784294 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.784231 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:27:45.784294 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.784246 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:27:45.784402 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.784323 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:27:45.784402 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.784341 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:27:45.784402 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.784351 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:27:45.785605 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.785591 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" Jul 06 19:27:45.785663 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.785617 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:27:45.786344 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.786328 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:27:45.786456 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.786353 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:27:45.786456 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.786383 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:27:45.805163 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.805138 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-33.ec2.internal\" not found" node="ip-10-0-141-33.ec2.internal" Jul 06 19:27:45.809996 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.809976 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-33.ec2.internal\" not found" node="ip-10-0-141-33.ec2.internal" Jul 06 19:27:45.862429 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.862393 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:45.900163 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.900130 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce1a1266be1a92ae6e1cacc841388c53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal\" (UID: \"ce1a1266be1a92ae6e1cacc841388c53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Jul 06 19:27:45.900320 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.900166 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce1a1266be1a92ae6e1cacc841388c53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal\" (UID: \"ce1a1266be1a92ae6e1cacc841388c53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Jul 06 19:27:45.900320 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:45.900187 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85b0a6ebff246af6adbee180d5545e92-config\") pod \"kube-apiserver-proxy-ip-10-0-141-33.ec2.internal\" (UID: \"85b0a6ebff246af6adbee180d5545e92\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" Jul 06 19:27:45.963242 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:45.963146 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:46.000446 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.000413 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85b0a6ebff246af6adbee180d5545e92-config\") pod \"kube-apiserver-proxy-ip-10-0-141-33.ec2.internal\" (UID: \"85b0a6ebff246af6adbee180d5545e92\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" Jul 06 19:27:46.000446 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.000447 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce1a1266be1a92ae6e1cacc841388c53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal\" (UID: \"ce1a1266be1a92ae6e1cacc841388c53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Jul 06 19:27:46.000628 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.000466 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce1a1266be1a92ae6e1cacc841388c53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal\" (UID: \"ce1a1266be1a92ae6e1cacc841388c53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Jul 06 19:27:46.000628 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.000522 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce1a1266be1a92ae6e1cacc841388c53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal\" (UID: \"ce1a1266be1a92ae6e1cacc841388c53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Jul 06 19:27:46.000628 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.000524 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85b0a6ebff246af6adbee180d5545e92-config\") pod \"kube-apiserver-proxy-ip-10-0-141-33.ec2.internal\" (UID: \"85b0a6ebff246af6adbee180d5545e92\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" Jul 06 19:27:46.000628 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.000595 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce1a1266be1a92ae6e1cacc841388c53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal\" (UID: \"ce1a1266be1a92ae6e1cacc841388c53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Jul 06 19:27:46.064167 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:46.064127 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:46.109057 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.109028 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Jul 06 19:27:46.114213 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.114192 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" Jul 06 19:27:46.164270 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:46.164228 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:46.269429 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:46.269346 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:46.371426 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:46.371382 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:46.432354 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.432290 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 19:27:46.433073 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.432486 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 19:27:46.472038 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:46.472010 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:46.573138 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:46.573094 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:46.594534 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.594494 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 19:22:46 +0000 UTC" deadline="2028-03-07 14:16:42.784653223 +0000 UTC" Jul 06 19:27:46.594534 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.594530 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14634h48m56.190128087s" Jul 06 19:27:46.639971 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.639945 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:27:46.646014 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:46.645905 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce1a1266be1a92ae6e1cacc841388c53.slice/crio-af2b63b926cac9b63701adeb568f975181be40274b11404d5fe7fb75eb7ee6e1 WatchSource:0}: Error finding container af2b63b926cac9b63701adeb568f975181be40274b11404d5fe7fb75eb7ee6e1: Status 404 returned error can't find the container with id af2b63b926cac9b63701adeb568f975181be40274b11404d5fe7fb75eb7ee6e1 Jul 06 19:27:46.646629 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:46.646587 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85b0a6ebff246af6adbee180d5545e92.slice/crio-a920344e5e8075b0aea79d7b2317f211798478f6a8531369d56d1502a86a7ad7 WatchSource:0}: Error finding container a920344e5e8075b0aea79d7b2317f211798478f6a8531369d56d1502a86a7ad7: Status 404 returned error can't find the container with id a920344e5e8075b0aea79d7b2317f211798478f6a8531369d56d1502a86a7ad7 Jul 06 19:27:46.653738 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.653718 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:27:46.662758 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.662737 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:27:46.663691 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.663671 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:27:46.663770 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.663705 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:27:46.663770 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.663717 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:27:46.673885 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:46.673864 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:46.683372 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.683323 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" event={"ID":"85b0a6ebff246af6adbee180d5545e92","Type":"ContainerStarted","Data":"a920344e5e8075b0aea79d7b2317f211798478f6a8531369d56d1502a86a7ad7"} Jul 06 19:27:46.684402 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.684380 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" event={"ID":"ce1a1266be1a92ae6e1cacc841388c53","Type":"ContainerStarted","Data":"af2b63b926cac9b63701adeb568f975181be40274b11404d5fe7fb75eb7ee6e1"} Jul 06 19:27:46.775019 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:46.774984 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:46.875482 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:46.875391 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:46.951647 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:46.951611 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:27:46.976553 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:46.976525 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:47.077383 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:47.077332 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-33.ec2.internal\" not found" Jul 06 19:27:47.104000 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.103977 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:27:47.200052 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.199942 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" Jul 06 19:27:47.230922 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.230891 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:27:47.232048 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.232021 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" Jul 06 19:27:47.245338 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.245311 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:27:47.476250 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.476164 2561 apiserver.go:52] "Watching apiserver" Jul 06 19:27:47.489846 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.489809 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 19:27:47.491986 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.491955 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-rk8nh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal","openshift-multus/multus-lmp4g","openshift-ovn-kubernetes/ovnkube-node-mnlq8","kube-system/global-pull-secret-syncer-ddpss","kube-system/konnectivity-agent-klbgj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7","openshift-dns/node-resolver-lg8j5","openshift-multus/multus-additional-cni-plugins-kdjjh","openshift-multus/network-metrics-daemon-26crl","openshift-network-diagnostics/network-check-target-28tgs","openshift-network-operator/iptables-alerter-7g78t","kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal","openshift-cluster-node-tuning-operator/tuned-rkqns"] Jul 06 19:27:47.494433 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.494409 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:47.494549 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:47.494511 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:27:47.496366 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.496328 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7g78t" Jul 06 19:27:47.498436 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.498411 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.500800 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.500715 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 19:27:47.500897 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.500802 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:27:47.500965 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.500916 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 19:27:47.501300 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.501153 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9hz74\"" Jul 06 19:27:47.501624 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.501502 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 19:27:47.501723 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.501675 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.503046 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.503025 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.505313 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.505292 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lg8j5" Jul 06 19:27:47.509360 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.509338 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.513472 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.513448 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514526 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6s68c\"" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.513588 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-slash\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514643 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b100a06c-0a61-478c-a28f-77630d8665c2-host-slash\") pod \"iptables-alerter-7g78t\" (UID: \"b100a06c-0a61-478c-a28f-77630d8665c2\") " pod="openshift-network-operator/iptables-alerter-7g78t" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514672 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-var-lib-cni-bin\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514698 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-var-lib-cni-multus\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514729 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-etc-kubernetes\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514743 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h99cc\"" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514756 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-run-openvswitch\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514781 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-log-socket\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514809 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs\") pod \"network-metrics-daemon-26crl\" (UID: \"477cd95b-0bfb-4470-8226-9ee5ed229f99\") " pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514837 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.513653 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514037 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514837 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2pzl\" (UniqueName: \"kubernetes.io/projected/b100a06c-0a61-478c-a28f-77630d8665c2-kube-api-access-w2pzl\") pod \"iptables-alerter-7g78t\" (UID: \"b100a06c-0a61-478c-a28f-77630d8665c2\") " pod="openshift-network-operator/iptables-alerter-7g78t" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.513604 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.515001 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-multus-socket-dir-parent\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.515027 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-run-k8s-cni-cncf-io\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.515020 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.515053 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-multus-daemon-config\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514066 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cck9p\"" Jul 06 19:27:47.515256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.515080 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-run-netns\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.515110 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.513702 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.515106 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-etc-openvswitch\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514080 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.513815 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514360 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sq7z5\"" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.513987 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.513916 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.515190 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-system-cni-dir\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.516908 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-run-netns\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.516938 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-var-lib-kubelet\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.516979 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-cni-netd\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514120 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.514379 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.517545 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:47.517639 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.517985 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-ovnkube-config\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518132 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-ovn-node-metrics-cert\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518166 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b100a06c-0a61-478c-a28f-77630d8665c2-iptables-alerter-script\") pod \"iptables-alerter-7g78t\" (UID: \"b100a06c-0a61-478c-a28f-77630d8665c2\") " pod="openshift-network-operator/iptables-alerter-7g78t" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518203 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-cnibin\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518235 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-hostroot\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.519234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518263 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfx28\" (UniqueName: \"kubernetes.io/projected/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-kube-api-access-wfx28\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518404 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5qzx\" (UniqueName: \"kubernetes.io/projected/477cd95b-0bfb-4470-8226-9ee5ed229f99-kube-api-access-z5qzx\") pod \"network-metrics-daemon-26crl\" (UID: \"477cd95b-0bfb-4470-8226-9ee5ed229f99\") " pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518438 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-cni-binary-copy\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518490 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-run-multus-certs\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518543 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-kubelet\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518623 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-run-ovn\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518661 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-node-log\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518694 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-cni-bin\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518722 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l49g7\" (UniqueName: \"kubernetes.io/projected/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-kube-api-access-l49g7\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518829 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-systemd-units\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518873 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-var-lib-openvswitch\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.518952 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-run-ovn-kubernetes\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.519013 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-os-release\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.519044 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-run-systemd\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.519132 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bbbsc\"" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.519197 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.519235 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-env-overrides\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.520310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.519267 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-ovnkube-script-lib\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.521220 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.519300 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-multus-cni-dir\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.521220 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.519317 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 19:27:47.521220 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.519326 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 19:27:47.521220 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.519333 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-multus-conf-dir\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.521220 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.519982 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rk8nh" Jul 06 19:27:47.525571 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.524414 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.526739 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.526721 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-klbgj" Jul 06 19:27:47.527503 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.527477 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 19:27:47.527916 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.527898 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 19:27:47.528912 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.528132 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 19:27:47.528912 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.528458 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z9rrt\"" Jul 06 19:27:47.528912 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.528468 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 19:27:47.528912 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.528688 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 19:27:47.528912 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.528763 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 19:27:47.529207 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.529074 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:47.529207 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:47.529143 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:27:47.529207 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.529077 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qvtqt\"" Jul 06 19:27:47.532655 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.532636 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 19:27:47.533613 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.533573 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gwvng\"" Jul 06 19:27:47.533885 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.533868 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 19:27:47.559450 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.559284 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:27:47.601286 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.601261 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 19:27:47.612732 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.612696 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vct4k" Jul 06 19:27:47.619967 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.619930 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-env-overrides\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.620129 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.619975 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-ovnkube-script-lib\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.620129 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620007 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-multus-cni-dir\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.620129 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620103 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-multus-cni-dir\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.620418 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620393 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-multus-conf-dir\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.620575 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620437 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/da99b45a-4a58-47d1-a98f-a05a8bb9801c-os-release\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.620575 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620468 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-sysctl-d\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.620575 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620497 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-tuned\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.620575 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620522 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc341ace-408b-47f4-8fbf-adaf13c4772f-hosts-file\") pod \"node-resolver-lg8j5\" (UID: \"bc341ace-408b-47f4-8fbf-adaf13c4772f\") " pod="openshift-dns/node-resolver-lg8j5" Jul 06 19:27:47.620575 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620547 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-sys-fs\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.620575 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620576 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-slash\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.620905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620602 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-var-lib-cni-bin\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.620905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620628 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-var-lib-cni-multus\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.620905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620657 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-log-socket\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.620905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620683 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-kubernetes\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.620905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620693 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-env-overrides\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.620905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620711 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da99b45a-4a58-47d1-a98f-a05a8bb9801c-cni-binary-copy\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.620905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620739 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs\") pod \"network-metrics-daemon-26crl\" (UID: \"477cd95b-0bfb-4470-8226-9ee5ed229f99\") " pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:47.620905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620767 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-run-netns\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.620905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620801 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-ovnkube-script-lib\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.620905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620802 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-multus-conf-dir\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.620905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620796 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-cni-netd\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.620905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620844 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-systemd\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.620905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620907 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-var-lib-cni-bin\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.620905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620913 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-run-netns\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620952 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-slash\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:47.620965 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620975 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-cni-netd\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.620998 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-var-lib-cni-multus\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621011 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-sys\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621042 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-log-socket\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:47.621056 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs podName:477cd95b-0bfb-4470-8226-9ee5ed229f99 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:48.121019493 +0000 UTC m=+2.988695047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs") pod "network-metrics-daemon-26crl" (UID: "477cd95b-0bfb-4470-8226-9ee5ed229f99") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621144 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-var-lib-kubelet\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621174 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-modprobe-d\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621202 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f154a1ff-5e8f-45b5-a443-b2e679c499d8-tmp\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621218 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-var-lib-kubelet\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621228 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-device-dir\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621257 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b100a06c-0a61-478c-a28f-77630d8665c2-iptables-alerter-script\") pod \"iptables-alerter-7g78t\" (UID: \"b100a06c-0a61-478c-a28f-77630d8665c2\") " pod="openshift-network-operator/iptables-alerter-7g78t" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621286 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da99b45a-4a58-47d1-a98f-a05a8bb9801c-system-cni-dir\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621334 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da99b45a-4a58-47d1-a98f-a05a8bb9801c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621376 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/766b167f-bf67-495e-86e7-1a06223c14ae-host\") pod \"node-ca-rk8nh\" (UID: \"766b167f-bf67-495e-86e7-1a06223c14ae\") " pod="openshift-image-registry/node-ca-rk8nh" Jul 06 19:27:47.621717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621418 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5qzx\" (UniqueName: \"kubernetes.io/projected/477cd95b-0bfb-4470-8226-9ee5ed229f99-kube-api-access-z5qzx\") pod \"network-metrics-daemon-26crl\" (UID: \"477cd95b-0bfb-4470-8226-9ee5ed229f99\") " pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621449 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-cni-binary-copy\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621477 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-kubelet\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621503 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-run-ovn\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621528 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-node-log\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621572 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-kubelet\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621613 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-cni-bin\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621640 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-node-log\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621641 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/766b167f-bf67-495e-86e7-1a06223c14ae-serviceca\") pod \"node-ca-rk8nh\" (UID: \"766b167f-bf67-495e-86e7-1a06223c14ae\") " pod="openshift-image-registry/node-ca-rk8nh" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621694 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-cni-bin\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621697 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-run-ovn\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621703 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc341ace-408b-47f4-8fbf-adaf13c4772f-tmp-dir\") pod \"node-resolver-lg8j5\" (UID: \"bc341ace-408b-47f4-8fbf-adaf13c4772f\") " pod="openshift-dns/node-resolver-lg8j5" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621744 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-var-lib-openvswitch\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621772 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-run-ovn-kubernetes\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621798 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-lib-modules\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621807 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-var-lib-openvswitch\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621826 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6lwl\" (UniqueName: \"kubernetes.io/projected/bc341ace-408b-47f4-8fbf-adaf13c4772f-kube-api-access-c6lwl\") pod \"node-resolver-lg8j5\" (UID: \"bc341ace-408b-47f4-8fbf-adaf13c4772f\") " pod="openshift-dns/node-resolver-lg8j5" Jul 06 19:27:47.622630 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621858 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-run-ovn-kubernetes\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621878 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b100a06c-0a61-478c-a28f-77630d8665c2-iptables-alerter-script\") pod \"iptables-alerter-7g78t\" (UID: \"b100a06c-0a61-478c-a28f-77630d8665c2\") " pod="openshift-network-operator/iptables-alerter-7g78t" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621899 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621929 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-socket-dir\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621957 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-registration-dir\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.621982 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-etc-selinux\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622010 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-os-release\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622035 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-run-systemd\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622064 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-cni-binary-copy\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622064 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622102 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622102 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-run-systemd\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622116 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/da99b45a-4a58-47d1-a98f-a05a8bb9801c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622125 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-os-release\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622148 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-host\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622175 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4d150905-fd23-4a84-9a45-729d76fd6652-dbus\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:47.623491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622208 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622230 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/da99b45a-4a58-47d1-a98f-a05a8bb9801c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622245 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-sysconfig\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622261 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d457j\" (UniqueName: \"kubernetes.io/projected/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-kube-api-access-d457j\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622285 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b100a06c-0a61-478c-a28f-77630d8665c2-host-slash\") pod \"iptables-alerter-7g78t\" (UID: \"b100a06c-0a61-478c-a28f-77630d8665c2\") " pod="openshift-network-operator/iptables-alerter-7g78t" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622309 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-etc-kubernetes\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622337 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-run-openvswitch\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622366 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b100a06c-0a61-478c-a28f-77630d8665c2-host-slash\") pod \"iptables-alerter-7g78t\" (UID: \"b100a06c-0a61-478c-a28f-77630d8665c2\") " pod="openshift-network-operator/iptables-alerter-7g78t" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622380 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkbtt\" (UniqueName: \"kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt\") pod \"network-check-target-28tgs\" (UID: \"46dc289d-2577-400d-84d5-8411d4225d46\") " pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622409 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2pzl\" (UniqueName: \"kubernetes.io/projected/b100a06c-0a61-478c-a28f-77630d8665c2-kube-api-access-w2pzl\") pod \"iptables-alerter-7g78t\" (UID: \"b100a06c-0a61-478c-a28f-77630d8665c2\") " pod="openshift-network-operator/iptables-alerter-7g78t" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622416 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-etc-kubernetes\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622446 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-multus-socket-dir-parent\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622474 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-run-k8s-cni-cncf-io\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622497 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-multus-daemon-config\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622520 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-etc-openvswitch\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622543 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-ovn-node-metrics-cert\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622570 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da99b45a-4a58-47d1-a98f-a05a8bb9801c-cnibin\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.624361 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622595 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-system-cni-dir\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622619 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-run-netns\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622646 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-ovnkube-config\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622667 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-run-openvswitch\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622674 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfx28\" (UniqueName: \"kubernetes.io/projected/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-kube-api-access-wfx28\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622701 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4d150905-fd23-4a84-9a45-729d76fd6652-kubelet-config\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622730 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-sysctl-conf\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622755 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-cnibin\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622779 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-hostroot\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622804 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/90559109-2258-4954-8b4c-c99a0e8aff98-agent-certs\") pod \"konnectivity-agent-klbgj\" (UID: \"90559109-2258-4954-8b4c-c99a0e8aff98\") " pod="kube-system/konnectivity-agent-klbgj" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622830 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-run-multus-certs\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622856 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-var-lib-kubelet\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622876 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-run-k8s-cni-cncf-io\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622963 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-cnibin\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623010 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-hostroot\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622997 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.622884 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l49g7\" (UniqueName: \"kubernetes.io/projected/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-kube-api-access-l49g7\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623072 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-multus-socket-dir-parent\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.625252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623075 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-systemd-units\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.626159 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623104 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/90559109-2258-4954-8b4c-c99a0e8aff98-konnectivity-ca\") pod \"konnectivity-agent-klbgj\" (UID: \"90559109-2258-4954-8b4c-c99a0e8aff98\") " pod="kube-system/konnectivity-agent-klbgj" Jul 06 19:27:47.626159 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623116 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-systemd-units\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.626159 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623149 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-run-multus-certs\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.626159 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623168 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-system-cni-dir\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.626159 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623181 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrwkq\" (UniqueName: \"kubernetes.io/projected/da99b45a-4a58-47d1-a98f-a05a8bb9801c-kube-api-access-qrwkq\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.626159 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623211 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-host-run-netns\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.626159 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623296 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxxxf\" (UniqueName: \"kubernetes.io/projected/766b167f-bf67-495e-86e7-1a06223c14ae-kube-api-access-nxxxf\") pod \"node-ca-rk8nh\" (UID: \"766b167f-bf67-495e-86e7-1a06223c14ae\") " pod="openshift-image-registry/node-ca-rk8nh" Jul 06 19:27:47.626159 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623361 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-etc-openvswitch\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.626159 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623449 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-run\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.626159 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623533 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzb8v\" (UniqueName: \"kubernetes.io/projected/f154a1ff-5e8f-45b5-a443-b2e679c499d8-kube-api-access-bzb8v\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.626159 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623722 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-multus-daemon-config\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.626159 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.623921 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-ovnkube-config\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.626841 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.626812 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-ovn-node-metrics-cert\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.646588 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.646550 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vct4k" Jul 06 19:27:47.654523 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.654441 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfx28\" (UniqueName: \"kubernetes.io/projected/3d23fd58-2dcd-4259-ad3f-a96d811ab0b7-kube-api-access-wfx28\") pod \"ovnkube-node-mnlq8\" (UID: \"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7\") " pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.668121 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.668086 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2pzl\" (UniqueName: \"kubernetes.io/projected/b100a06c-0a61-478c-a28f-77630d8665c2-kube-api-access-w2pzl\") pod \"iptables-alerter-7g78t\" (UID: \"b100a06c-0a61-478c-a28f-77630d8665c2\") " pod="openshift-network-operator/iptables-alerter-7g78t" Jul 06 19:27:47.700792 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.700756 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5qzx\" (UniqueName: \"kubernetes.io/projected/477cd95b-0bfb-4470-8226-9ee5ed229f99-kube-api-access-z5qzx\") pod \"network-metrics-daemon-26crl\" (UID: \"477cd95b-0bfb-4470-8226-9ee5ed229f99\") " pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:47.702858 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.702828 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l49g7\" (UniqueName: \"kubernetes.io/projected/62bc027b-c11e-4a1b-80e3-68dc5d8f8441-kube-api-access-l49g7\") pod \"multus-lmp4g\" (UID: \"62bc027b-c11e-4a1b-80e3-68dc5d8f8441\") " pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.724683 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.724643 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:47.724892 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:47.724793 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:27:47.724892 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.724808 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/da99b45a-4a58-47d1-a98f-a05a8bb9801c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.724892 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.724853 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-sysconfig\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.724892 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:47.724875 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret podName:4d150905-fd23-4a84-9a45-729d76fd6652 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:48.224853574 +0000 UTC m=+3.092529110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret") pod "global-pull-secret-syncer-ddpss" (UID: "4d150905-fd23-4a84-9a45-729d76fd6652") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:27:47.725126 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.724910 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d457j\" (UniqueName: \"kubernetes.io/projected/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-kube-api-access-d457j\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.725126 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.724924 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-sysconfig\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.725126 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.724952 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkbtt\" (UniqueName: \"kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt\") pod \"network-check-target-28tgs\" (UID: \"46dc289d-2577-400d-84d5-8411d4225d46\") " pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:27:47.725126 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.724987 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da99b45a-4a58-47d1-a98f-a05a8bb9801c-cnibin\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.725126 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725022 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4d150905-fd23-4a84-9a45-729d76fd6652-kubelet-config\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:47.725126 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725083 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4d150905-fd23-4a84-9a45-729d76fd6652-kubelet-config\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725130 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da99b45a-4a58-47d1-a98f-a05a8bb9801c-cnibin\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725240 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-sysctl-conf\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725282 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/90559109-2258-4954-8b4c-c99a0e8aff98-agent-certs\") pod \"konnectivity-agent-klbgj\" (UID: \"90559109-2258-4954-8b4c-c99a0e8aff98\") " pod="kube-system/konnectivity-agent-klbgj" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725328 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-var-lib-kubelet\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725359 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/90559109-2258-4954-8b4c-c99a0e8aff98-konnectivity-ca\") pod \"konnectivity-agent-klbgj\" (UID: \"90559109-2258-4954-8b4c-c99a0e8aff98\") " pod="kube-system/konnectivity-agent-klbgj" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725454 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-var-lib-kubelet\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725472 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/da99b45a-4a58-47d1-a98f-a05a8bb9801c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725484 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-sysctl-conf\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725501 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrwkq\" (UniqueName: \"kubernetes.io/projected/da99b45a-4a58-47d1-a98f-a05a8bb9801c-kube-api-access-qrwkq\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725560 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxxxf\" (UniqueName: \"kubernetes.io/projected/766b167f-bf67-495e-86e7-1a06223c14ae-kube-api-access-nxxxf\") pod \"node-ca-rk8nh\" (UID: \"766b167f-bf67-495e-86e7-1a06223c14ae\") " pod="openshift-image-registry/node-ca-rk8nh" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725587 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-run\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725615 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzb8v\" (UniqueName: \"kubernetes.io/projected/f154a1ff-5e8f-45b5-a443-b2e679c499d8-kube-api-access-bzb8v\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725645 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/da99b45a-4a58-47d1-a98f-a05a8bb9801c-os-release\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725670 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-sysctl-d\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725713 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-tuned\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725740 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc341ace-408b-47f4-8fbf-adaf13c4772f-hosts-file\") pod \"node-resolver-lg8j5\" (UID: \"bc341ace-408b-47f4-8fbf-adaf13c4772f\") " pod="openshift-dns/node-resolver-lg8j5" Jul 06 19:27:47.726859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725766 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-sys-fs\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725795 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-kubernetes\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725822 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da99b45a-4a58-47d1-a98f-a05a8bb9801c-cni-binary-copy\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725836 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-run\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725870 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-sysctl-d\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725896 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc341ace-408b-47f4-8fbf-adaf13c4772f-hosts-file\") pod \"node-resolver-lg8j5\" (UID: \"bc341ace-408b-47f4-8fbf-adaf13c4772f\") " pod="openshift-dns/node-resolver-lg8j5" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725913 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/da99b45a-4a58-47d1-a98f-a05a8bb9801c-os-release\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725927 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/90559109-2258-4954-8b4c-c99a0e8aff98-konnectivity-ca\") pod \"konnectivity-agent-klbgj\" (UID: \"90559109-2258-4954-8b4c-c99a0e8aff98\") " pod="kube-system/konnectivity-agent-klbgj" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725939 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-sys-fs\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.725969 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-kubernetes\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726317 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-systemd\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726342 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-sys\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726389 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-modprobe-d\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726414 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f154a1ff-5e8f-45b5-a443-b2e679c499d8-tmp\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726439 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-device-dir\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726465 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da99b45a-4a58-47d1-a98f-a05a8bb9801c-system-cni-dir\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726491 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da99b45a-4a58-47d1-a98f-a05a8bb9801c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.727761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726491 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da99b45a-4a58-47d1-a98f-a05a8bb9801c-cni-binary-copy\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726517 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/766b167f-bf67-495e-86e7-1a06223c14ae-host\") pod \"node-ca-rk8nh\" (UID: \"766b167f-bf67-495e-86e7-1a06223c14ae\") " pod="openshift-image-registry/node-ca-rk8nh" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726549 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/766b167f-bf67-495e-86e7-1a06223c14ae-serviceca\") pod \"node-ca-rk8nh\" (UID: \"766b167f-bf67-495e-86e7-1a06223c14ae\") " pod="openshift-image-registry/node-ca-rk8nh" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc341ace-408b-47f4-8fbf-adaf13c4772f-tmp-dir\") pod \"node-resolver-lg8j5\" (UID: \"bc341ace-408b-47f4-8fbf-adaf13c4772f\") " pod="openshift-dns/node-resolver-lg8j5" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726598 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-lib-modules\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726641 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6lwl\" (UniqueName: \"kubernetes.io/projected/bc341ace-408b-47f4-8fbf-adaf13c4772f-kube-api-access-c6lwl\") pod \"node-resolver-lg8j5\" (UID: \"bc341ace-408b-47f4-8fbf-adaf13c4772f\") " pod="openshift-dns/node-resolver-lg8j5" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726667 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726692 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-socket-dir\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726717 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-registration-dir\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726742 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-etc-selinux\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726774 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/da99b45a-4a58-47d1-a98f-a05a8bb9801c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726799 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-host\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726823 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4d150905-fd23-4a84-9a45-729d76fd6652-dbus\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726842 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-systemd\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726917 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-device-dir\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726967 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-modprobe-d\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.726982 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da99b45a-4a58-47d1-a98f-a05a8bb9801c-system-cni-dir\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.728587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.727062 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.729196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.727084 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4d150905-fd23-4a84-9a45-729d76fd6652-dbus\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:47.729196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.727086 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-etc-selinux\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.729196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.727140 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da99b45a-4a58-47d1-a98f-a05a8bb9801c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.729196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.727188 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-lib-modules\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.729196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.727336 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-socket-dir\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.729196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.727423 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc341ace-408b-47f4-8fbf-adaf13c4772f-tmp-dir\") pod \"node-resolver-lg8j5\" (UID: \"bc341ace-408b-47f4-8fbf-adaf13c4772f\") " pod="openshift-dns/node-resolver-lg8j5" Jul 06 19:27:47.729196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.727535 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-registration-dir\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.729196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.727592 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-sys\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.729196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.727611 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f154a1ff-5e8f-45b5-a443-b2e679c499d8-host\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.729196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.727610 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/766b167f-bf67-495e-86e7-1a06223c14ae-serviceca\") pod \"node-ca-rk8nh\" (UID: \"766b167f-bf67-495e-86e7-1a06223c14ae\") " pod="openshift-image-registry/node-ca-rk8nh" Jul 06 19:27:47.729196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.727652 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/da99b45a-4a58-47d1-a98f-a05a8bb9801c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.729196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.727675 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/766b167f-bf67-495e-86e7-1a06223c14ae-host\") pod \"node-ca-rk8nh\" (UID: \"766b167f-bf67-495e-86e7-1a06223c14ae\") " pod="openshift-image-registry/node-ca-rk8nh" Jul 06 19:27:47.729196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.728561 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/90559109-2258-4954-8b4c-c99a0e8aff98-agent-certs\") pod \"konnectivity-agent-klbgj\" (UID: \"90559109-2258-4954-8b4c-c99a0e8aff98\") " pod="kube-system/konnectivity-agent-klbgj" Jul 06 19:27:47.729779 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.729416 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f154a1ff-5e8f-45b5-a443-b2e679c499d8-etc-tuned\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.729779 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.729502 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f154a1ff-5e8f-45b5-a443-b2e679c499d8-tmp\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.754785 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:47.754747 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:27:47.754987 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:47.754812 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:27:47.754987 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:47.754830 2561 projected.go:196] Error preparing data for projected volume kube-api-access-vkbtt for pod openshift-network-diagnostics/network-check-target-28tgs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:27:47.754987 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:47.754922 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt podName:46dc289d-2577-400d-84d5-8411d4225d46 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:48.254899133 +0000 UTC m=+3.122574693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vkbtt" (UniqueName: "kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt") pod "network-check-target-28tgs" (UID: "46dc289d-2577-400d-84d5-8411d4225d46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:27:47.759426 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.759392 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxxxf\" (UniqueName: \"kubernetes.io/projected/766b167f-bf67-495e-86e7-1a06223c14ae-kube-api-access-nxxxf\") pod \"node-ca-rk8nh\" (UID: \"766b167f-bf67-495e-86e7-1a06223c14ae\") " pod="openshift-image-registry/node-ca-rk8nh" Jul 06 19:27:47.760774 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.760703 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6lwl\" (UniqueName: \"kubernetes.io/projected/bc341ace-408b-47f4-8fbf-adaf13c4772f-kube-api-access-c6lwl\") pod \"node-resolver-lg8j5\" (UID: \"bc341ace-408b-47f4-8fbf-adaf13c4772f\") " pod="openshift-dns/node-resolver-lg8j5" Jul 06 19:27:47.760908 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.760853 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzb8v\" (UniqueName: \"kubernetes.io/projected/f154a1ff-5e8f-45b5-a443-b2e679c499d8-kube-api-access-bzb8v\") pod \"tuned-rkqns\" (UID: \"f154a1ff-5e8f-45b5-a443-b2e679c499d8\") " pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.762014 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.761985 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrwkq\" (UniqueName: \"kubernetes.io/projected/da99b45a-4a58-47d1-a98f-a05a8bb9801c-kube-api-access-qrwkq\") pod \"multus-additional-cni-plugins-kdjjh\" (UID: \"da99b45a-4a58-47d1-a98f-a05a8bb9801c\") " pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.762461 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.762435 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d457j\" (UniqueName: \"kubernetes.io/projected/cd5fa48a-ec49-459e-8748-6ac2dc6c49e7-kube-api-access-d457j\") pod \"aws-ebs-csi-driver-node-rd6b7\" (UID: \"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.814623 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.814585 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7g78t" Jul 06 19:27:47.835506 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.835473 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lmp4g" Jul 06 19:27:47.848009 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.847978 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:27:47.855220 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.855191 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rkqns" Jul 06 19:27:47.863787 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.863761 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lg8j5" Jul 06 19:27:47.870886 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.870863 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:27:47.871968 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.871946 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kdjjh" Jul 06 19:27:47.881157 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.881130 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rk8nh" Jul 06 19:27:47.890248 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.890218 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" Jul 06 19:27:47.896267 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:47.896237 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-klbgj" Jul 06 19:27:48.130887 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:48.130786 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs\") pod \"network-metrics-daemon-26crl\" (UID: \"477cd95b-0bfb-4470-8226-9ee5ed229f99\") " pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:48.131072 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:48.130920 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:27:48.131072 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:48.130984 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs podName:477cd95b-0bfb-4470-8226-9ee5ed229f99 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:49.130969816 +0000 UTC m=+3.998645352 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs") pod "network-metrics-daemon-26crl" (UID: "477cd95b-0bfb-4470-8226-9ee5ed229f99") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:27:48.231946 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:48.231912 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:48.232117 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:48.232078 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:27:48.232168 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:48.232145 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret podName:4d150905-fd23-4a84-9a45-729d76fd6652 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:49.232128964 +0000 UTC m=+4.099804506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret") pod "global-pull-secret-syncer-ddpss" (UID: "4d150905-fd23-4a84-9a45-729d76fd6652") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:27:48.332357 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:48.332306 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkbtt\" (UniqueName: \"kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt\") pod \"network-check-target-28tgs\" (UID: \"46dc289d-2577-400d-84d5-8411d4225d46\") " pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:27:48.332540 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:48.332505 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:27:48.332540 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:48.332529 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:27:48.332648 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:48.332547 2561 projected.go:196] Error preparing data for projected volume kube-api-access-vkbtt for pod openshift-network-diagnostics/network-check-target-28tgs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:27:48.332648 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:48.332619 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt podName:46dc289d-2577-400d-84d5-8411d4225d46 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:49.332598226 +0000 UTC m=+4.200273781 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vkbtt" (UniqueName: "kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt") pod "network-check-target-28tgs" (UID: "46dc289d-2577-400d-84d5-8411d4225d46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:27:48.540409 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:48.540337 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb100a06c_0a61_478c_a28f_77630d8665c2.slice/crio-0c1d41c5eb5b741414017e9848d13ead839d5da079a9ee42be9f8525ba2d964e WatchSource:0}: Error finding container 0c1d41c5eb5b741414017e9848d13ead839d5da079a9ee42be9f8525ba2d964e: Status 404 returned error can't find the container with id 0c1d41c5eb5b741414017e9848d13ead839d5da079a9ee42be9f8525ba2d964e Jul 06 19:27:48.651545 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:48.651491 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:22:48 +0000 UTC" deadline="2028-04-03 10:41:56.803553846 +0000 UTC" Jul 06 19:27:48.651545 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:48.651540 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15279h14m8.152018005s" Jul 06 19:27:48.680844 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:48.680798 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:27:48.681034 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:48.680927 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:27:48.688508 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:48.688459 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7g78t" event={"ID":"b100a06c-0a61-478c-a28f-77630d8665c2","Type":"ContainerStarted","Data":"0c1d41c5eb5b741414017e9848d13ead839d5da079a9ee42be9f8525ba2d964e"} Jul 06 19:27:48.947383 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:48.947139 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd5fa48a_ec49_459e_8748_6ac2dc6c49e7.slice/crio-1fdd63130daa4da3ab48b5517c333866c563f2e43f5c3ae490eefe6b4ed7300b WatchSource:0}: Error finding container 1fdd63130daa4da3ab48b5517c333866c563f2e43f5c3ae490eefe6b4ed7300b: Status 404 returned error can't find the container with id 1fdd63130daa4da3ab48b5517c333866c563f2e43f5c3ae490eefe6b4ed7300b Jul 06 19:27:48.948907 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:48.948869 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda99b45a_4a58_47d1_a98f_a05a8bb9801c.slice/crio-4a304f910df7daff68e1af2ce28cad0d2565de19c3bde86dba5f68740cbdb60d WatchSource:0}: Error finding container 4a304f910df7daff68e1af2ce28cad0d2565de19c3bde86dba5f68740cbdb60d: Status 404 returned error can't find the container with id 4a304f910df7daff68e1af2ce28cad0d2565de19c3bde86dba5f68740cbdb60d Jul 06 19:27:48.952534 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:48.952505 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62bc027b_c11e_4a1b_80e3_68dc5d8f8441.slice/crio-5fd79cde76a451c902a27f8836b1e1c3e87997395d7311a4be964c0c515d58a2 WatchSource:0}: Error finding container 5fd79cde76a451c902a27f8836b1e1c3e87997395d7311a4be964c0c515d58a2: Status 404 returned error can't find the container with id 5fd79cde76a451c902a27f8836b1e1c3e87997395d7311a4be964c0c515d58a2 Jul 06 19:27:48.954192 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:48.954030 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90559109_2258_4954_8b4c_c99a0e8aff98.slice/crio-81b05bf64b7437d42e184ebb7cfd1f9c8b1c83ae9e97edb588bed11e8f1bf3cc WatchSource:0}: Error finding container 81b05bf64b7437d42e184ebb7cfd1f9c8b1c83ae9e97edb588bed11e8f1bf3cc: Status 404 returned error can't find the container with id 81b05bf64b7437d42e184ebb7cfd1f9c8b1c83ae9e97edb588bed11e8f1bf3cc Jul 06 19:27:48.954322 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:48.954293 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod766b167f_bf67_495e_86e7_1a06223c14ae.slice/crio-e4f86379b601146654a9bc8bb7a6c40c90066c541237fbee47c2e1a61b710ec3 WatchSource:0}: Error finding container e4f86379b601146654a9bc8bb7a6c40c90066c541237fbee47c2e1a61b710ec3: Status 404 returned error can't find the container with id e4f86379b601146654a9bc8bb7a6c40c90066c541237fbee47c2e1a61b710ec3 Jul 06 19:27:48.956524 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:48.956285 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf154a1ff_5e8f_45b5_a443_b2e679c499d8.slice/crio-a77b2928d8975cef4df35f73b9eff2f0f0593008ffc1191d634d6c2a320abb20 WatchSource:0}: Error finding container a77b2928d8975cef4df35f73b9eff2f0f0593008ffc1191d634d6c2a320abb20: Status 404 returned error can't find the container with id a77b2928d8975cef4df35f73b9eff2f0f0593008ffc1191d634d6c2a320abb20 Jul 06 19:27:48.957114 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:27:48.957069 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc341ace_408b_47f4_8fbf_adaf13c4772f.slice/crio-6db93c788ef2f73103ed047c514482ed63a9f38913ce57b21e1204c3da092b3d WatchSource:0}: Error finding container 6db93c788ef2f73103ed047c514482ed63a9f38913ce57b21e1204c3da092b3d: Status 404 returned error can't find the container with id 6db93c788ef2f73103ed047c514482ed63a9f38913ce57b21e1204c3da092b3d Jul 06 19:27:49.140707 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.140661 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs\") pod \"network-metrics-daemon-26crl\" (UID: \"477cd95b-0bfb-4470-8226-9ee5ed229f99\") " pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:49.140921 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:49.140898 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:27:49.141019 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:49.140983 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs podName:477cd95b-0bfb-4470-8226-9ee5ed229f99 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:51.140961359 +0000 UTC m=+6.008636896 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs") pod "network-metrics-daemon-26crl" (UID: "477cd95b-0bfb-4470-8226-9ee5ed229f99") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:27:49.241759 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.241722 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:49.241899 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:49.241877 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:27:49.241965 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:49.241953 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret podName:4d150905-fd23-4a84-9a45-729d76fd6652 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:51.241933367 +0000 UTC m=+6.109608909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret") pod "global-pull-secret-syncer-ddpss" (UID: "4d150905-fd23-4a84-9a45-729d76fd6652") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:27:49.343044 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.343004 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkbtt\" (UniqueName: \"kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt\") pod \"network-check-target-28tgs\" (UID: \"46dc289d-2577-400d-84d5-8411d4225d46\") " pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:27:49.343245 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:49.343221 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:27:49.343310 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:49.343258 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:27:49.343310 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:49.343274 2561 projected.go:196] Error preparing data for projected volume kube-api-access-vkbtt for pod openshift-network-diagnostics/network-check-target-28tgs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:27:49.343415 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:49.343355 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt podName:46dc289d-2577-400d-84d5-8411d4225d46 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:51.343336303 +0000 UTC m=+6.211011841 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vkbtt" (UniqueName: "kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt") pod "network-check-target-28tgs" (UID: "46dc289d-2577-400d-84d5-8411d4225d46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:27:49.652375 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.652285 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:22:48 +0000 UTC" deadline="2028-04-11 13:54:19.95000806 +0000 UTC" Jul 06 19:27:49.652375 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.652373 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15474h26m30.297640052s" Jul 06 19:27:49.683682 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.683553 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:49.683879 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:49.683708 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:27:49.684138 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.684111 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:49.684225 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:49.684207 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:27:49.699871 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.699781 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rkqns" event={"ID":"f154a1ff-5e8f-45b5-a443-b2e679c499d8","Type":"ContainerStarted","Data":"a77b2928d8975cef4df35f73b9eff2f0f0593008ffc1191d634d6c2a320abb20"} Jul 06 19:27:49.704244 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.704158 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-klbgj" event={"ID":"90559109-2258-4954-8b4c-c99a0e8aff98","Type":"ContainerStarted","Data":"81b05bf64b7437d42e184ebb7cfd1f9c8b1c83ae9e97edb588bed11e8f1bf3cc"} Jul 06 19:27:49.705988 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.705906 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kdjjh" event={"ID":"da99b45a-4a58-47d1-a98f-a05a8bb9801c","Type":"ContainerStarted","Data":"4a304f910df7daff68e1af2ce28cad0d2565de19c3bde86dba5f68740cbdb60d"} Jul 06 19:27:49.709810 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.709712 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" event={"ID":"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7","Type":"ContainerStarted","Data":"1fdd63130daa4da3ab48b5517c333866c563f2e43f5c3ae490eefe6b4ed7300b"} Jul 06 19:27:49.719009 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.718302 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" event={"ID":"85b0a6ebff246af6adbee180d5545e92","Type":"ContainerStarted","Data":"151a9a2de795546024254046c6d82498244ef97db32d3778e6fed4bc7b5bcb70"} Jul 06 19:27:49.725135 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.724957 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" event={"ID":"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7","Type":"ContainerStarted","Data":"535e0bae173da23c5227a584e0f9a9dd7f764f68cab123bfbd553268cd4096dd"} Jul 06 19:27:49.733623 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.733582 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rk8nh" event={"ID":"766b167f-bf67-495e-86e7-1a06223c14ae","Type":"ContainerStarted","Data":"e4f86379b601146654a9bc8bb7a6c40c90066c541237fbee47c2e1a61b710ec3"} Jul 06 19:27:49.745787 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.745747 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lmp4g" event={"ID":"62bc027b-c11e-4a1b-80e3-68dc5d8f8441","Type":"ContainerStarted","Data":"5fd79cde76a451c902a27f8836b1e1c3e87997395d7311a4be964c0c515d58a2"} Jul 06 19:27:49.775972 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.775930 2561 generic.go:356] "Generic (PLEG): container finished" podID="ce1a1266be1a92ae6e1cacc841388c53" containerID="3d3136ded8e33107cd178d041da73909a709bdc5edf944d8da4b7b9768a70ae7" exitCode=0 Jul 06 19:27:49.776316 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.776264 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" event={"ID":"ce1a1266be1a92ae6e1cacc841388c53","Type":"ContainerDied","Data":"3d3136ded8e33107cd178d041da73909a709bdc5edf944d8da4b7b9768a70ae7"} Jul 06 19:27:49.789335 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.789267 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lg8j5" event={"ID":"bc341ace-408b-47f4-8fbf-adaf13c4772f","Type":"ContainerStarted","Data":"6db93c788ef2f73103ed047c514482ed63a9f38913ce57b21e1204c3da092b3d"} Jul 06 19:27:49.801036 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:49.800456 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-33.ec2.internal" podStartSLOduration=1.800430558 podStartE2EDuration="1.800430558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:27:49.744484463 +0000 UTC m=+4.612160023" watchObservedRunningTime="2026-07-06 19:27:49.800430558 +0000 UTC m=+4.668106133" Jul 06 19:27:50.681995 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:50.681492 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:27:50.681995 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:50.681622 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:27:50.801274 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:50.801234 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" event={"ID":"ce1a1266be1a92ae6e1cacc841388c53","Type":"ContainerStarted","Data":"43f451cc438536a8ddebd49eea94b2bfc41a81be9128de94ee88e28478638d22"} Jul 06 19:27:51.163103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:51.162345 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs\") pod \"network-metrics-daemon-26crl\" (UID: \"477cd95b-0bfb-4470-8226-9ee5ed229f99\") " pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:51.163103 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:51.162588 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:27:51.163103 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:51.162656 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs podName:477cd95b-0bfb-4470-8226-9ee5ed229f99 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:55.162636731 +0000 UTC m=+10.030312271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs") pod "network-metrics-daemon-26crl" (UID: "477cd95b-0bfb-4470-8226-9ee5ed229f99") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:27:51.264256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:51.263584 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:51.264256 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:51.263806 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:27:51.264256 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:51.263879 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret podName:4d150905-fd23-4a84-9a45-729d76fd6652 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:55.263854934 +0000 UTC m=+10.131530488 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret") pod "global-pull-secret-syncer-ddpss" (UID: "4d150905-fd23-4a84-9a45-729d76fd6652") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:27:51.364966 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:51.364924 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkbtt\" (UniqueName: \"kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt\") pod \"network-check-target-28tgs\" (UID: \"46dc289d-2577-400d-84d5-8411d4225d46\") " pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:27:51.365173 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:51.365152 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:27:51.365239 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:51.365184 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:27:51.365239 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:51.365200 2561 projected.go:196] Error preparing data for projected volume kube-api-access-vkbtt for pod openshift-network-diagnostics/network-check-target-28tgs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:27:51.365546 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:51.365268 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt podName:46dc289d-2577-400d-84d5-8411d4225d46 nodeName:}" failed. No retries permitted until 2026-07-06 19:27:55.36524438 +0000 UTC m=+10.232920158 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vkbtt" (UniqueName: "kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt") pod "network-check-target-28tgs" (UID: "46dc289d-2577-400d-84d5-8411d4225d46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:27:51.681499 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:51.681463 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:51.681679 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:51.681623 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:27:51.682444 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:51.682206 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:51.682444 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:51.682349 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:27:52.681564 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:52.681520 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:27:52.681754 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:52.681639 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:27:53.681496 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:53.681462 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:53.682006 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:53.681609 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:27:53.682181 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:53.682159 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:53.682315 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:53.682286 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:27:54.681450 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:54.681333 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:27:54.681678 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:54.681490 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:27:55.199509 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:55.199437 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs\") pod \"network-metrics-daemon-26crl\" (UID: \"477cd95b-0bfb-4470-8226-9ee5ed229f99\") " pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:55.199730 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:55.199624 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:27:55.199730 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:55.199694 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs podName:477cd95b-0bfb-4470-8226-9ee5ed229f99 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:03.199673136 +0000 UTC m=+18.067348674 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs") pod "network-metrics-daemon-26crl" (UID: "477cd95b-0bfb-4470-8226-9ee5ed229f99") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:27:55.300176 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:55.300131 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:55.300410 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:55.300321 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:27:55.300410 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:55.300403 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret podName:4d150905-fd23-4a84-9a45-729d76fd6652 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:03.300380145 +0000 UTC m=+18.168055699 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret") pod "global-pull-secret-syncer-ddpss" (UID: "4d150905-fd23-4a84-9a45-729d76fd6652") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:27:55.401058 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:55.401012 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkbtt\" (UniqueName: \"kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt\") pod \"network-check-target-28tgs\" (UID: \"46dc289d-2577-400d-84d5-8411d4225d46\") " pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:27:55.401254 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:55.401203 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:27:55.401254 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:55.401234 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:27:55.401254 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:55.401249 2561 projected.go:196] Error preparing data for projected volume kube-api-access-vkbtt for pod openshift-network-diagnostics/network-check-target-28tgs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:27:55.401476 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:55.401335 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt podName:46dc289d-2577-400d-84d5-8411d4225d46 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:03.401296013 +0000 UTC m=+18.268971570 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vkbtt" (UniqueName: "kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt") pod "network-check-target-28tgs" (UID: "46dc289d-2577-400d-84d5-8411d4225d46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:27:55.681777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:55.681674 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:55.682240 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:55.681819 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:27:55.682240 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:55.681882 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:55.682240 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:55.682011 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:27:56.680621 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:56.680581 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:27:56.680787 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:56.680704 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:27:57.685252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:57.685178 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:57.685252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:57.685195 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:57.685809 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:57.685307 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:27:57.685809 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:57.685451 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:27:58.681177 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:58.681120 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:27:58.681358 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:58.681256 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:27:59.688504 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:59.688237 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:27:59.688907 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:59.688612 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:27:59.688907 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:59.688326 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:27:59.688907 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:27:59.688770 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:27:59.831761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:59.831720 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lg8j5" event={"ID":"bc341ace-408b-47f4-8fbf-adaf13c4772f","Type":"ContainerStarted","Data":"14e0f672954a5418329303873f44b3ed3c138ac9d5bcc6406e092f6b9e3d4aa2"} Jul 06 19:27:59.852796 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:59.852733 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-33.ec2.internal" podStartSLOduration=11.852685171 podStartE2EDuration="11.852685171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:27:50.910866315 +0000 UTC m=+5.778541876" watchObservedRunningTime="2026-07-06 19:27:59.852685171 +0000 UTC m=+14.720360710" Jul 06 19:27:59.853703 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:27:59.853658 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lg8j5" podStartSLOduration=3.285847854 podStartE2EDuration="13.853639007s" totalImagesPullingTime="10.567791153s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:27:59.853523514 +0000 UTC m=+14.721199071" watchObservedRunningTime="2026-07-06 19:27:59.853639007 +0000 UTC m=+14.721314567" Jul 06 19:28:00.680667 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:00.680627 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:00.680862 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:00.680769 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:00.835305 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:00.835257 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rkqns" event={"ID":"f154a1ff-5e8f-45b5-a443-b2e679c499d8","Type":"ContainerStarted","Data":"bbc05719deafea81c64de269442874872247f824fcf2b7bd318f919b078ab74b"} Jul 06 19:28:00.836831 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:00.836802 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-klbgj" event={"ID":"90559109-2258-4954-8b4c-c99a0e8aff98","Type":"ContainerStarted","Data":"34f98276cf4d5200917303ab55c84c753ea5163b054823bee048309cc13fd4f6"} Jul 06 19:28:00.838501 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:00.838471 2561 generic.go:356] "Generic (PLEG): container finished" podID="da99b45a-4a58-47d1-a98f-a05a8bb9801c" containerID="77a12588aa1b049e782f853d54f8b709eafb314242525341e1fc972262479445" exitCode=0 Jul 06 19:28:00.838607 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:00.838555 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kdjjh" event={"ID":"da99b45a-4a58-47d1-a98f-a05a8bb9801c","Type":"ContainerDied","Data":"77a12588aa1b049e782f853d54f8b709eafb314242525341e1fc972262479445"} Jul 06 19:28:00.840239 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:00.840216 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" event={"ID":"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7","Type":"ContainerStarted","Data":"221120870e481fa209bfdd50ff050014057ebc12fdac5418d2a3dd6f0406eef9"} Jul 06 19:28:00.841872 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:00.841848 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rk8nh" event={"ID":"766b167f-bf67-495e-86e7-1a06223c14ae","Type":"ContainerStarted","Data":"dbe1f2c6883d0996ef45085836d55c692376fe24c56fbd125733a7e12c9250ce"} Jul 06 19:28:00.879310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:00.879253 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rkqns" podStartSLOduration=3.958440157 podStartE2EDuration="14.879235934s" totalImagesPullingTime="10.920795777s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:00.854917535 +0000 UTC m=+15.722593092" watchObservedRunningTime="2026-07-06 19:28:00.879235934 +0000 UTC m=+15.746911490" Jul 06 19:28:00.898395 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:00.898283 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rk8nh" podStartSLOduration=3.305123256 podStartE2EDuration="13.898267306s" totalImagesPullingTime="10.59314405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:00.898247712 +0000 UTC m=+15.765923268" watchObservedRunningTime="2026-07-06 19:28:00.898267306 +0000 UTC m=+15.765942862" Jul 06 19:28:00.916184 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:00.916131 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-klbgj" podStartSLOduration=3.321880913 podStartE2EDuration="13.916113894s" totalImagesPullingTime="10.594232981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:00.915597982 +0000 UTC m=+15.783273542" watchObservedRunningTime="2026-07-06 19:28:00.916113894 +0000 UTC m=+15.783789457" Jul 06 19:28:01.681504 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:01.681468 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:01.681678 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:01.681469 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:01.681678 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:01.681606 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:01.681770 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:01.681678 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:01.845492 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:01.845429 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7g78t" event={"ID":"b100a06c-0a61-478c-a28f-77630d8665c2","Type":"ContainerStarted","Data":"32c9ebee0169bf8b6d3172a09bdc2180384c71be70b5d1d8c5412c7537cecc3c"} Jul 06 19:28:01.864345 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:01.864276 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7g78t" podStartSLOduration=4.879060928 podStartE2EDuration="15.864260439s" totalImagesPullingTime="10.985199511s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:01.863546084 +0000 UTC m=+16.731221642" watchObservedRunningTime="2026-07-06 19:28:01.864260439 +0000 UTC m=+16.731935995" Jul 06 19:28:02.388902 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:02.388862 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-klbgj" Jul 06 19:28:02.389952 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:02.389919 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-klbgj" Jul 06 19:28:02.681154 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:02.681123 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:02.681357 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:02.681235 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:02.847772 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:02.847729 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-klbgj" Jul 06 19:28:02.848200 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:02.848086 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-klbgj" Jul 06 19:28:03.268661 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:03.268617 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs\") pod \"network-metrics-daemon-26crl\" (UID: \"477cd95b-0bfb-4470-8226-9ee5ed229f99\") " pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:03.268833 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:03.268794 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:28:03.268888 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:03.268866 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs podName:477cd95b-0bfb-4470-8226-9ee5ed229f99 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:19.268850429 +0000 UTC m=+34.136525963 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs") pod "network-metrics-daemon-26crl" (UID: "477cd95b-0bfb-4470-8226-9ee5ed229f99") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:28:03.369838 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:03.369802 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:03.370027 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:03.369918 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:28:03.370027 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:03.370001 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret podName:4d150905-fd23-4a84-9a45-729d76fd6652 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:19.369980904 +0000 UTC m=+34.237656450 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret") pod "global-pull-secret-syncer-ddpss" (UID: "4d150905-fd23-4a84-9a45-729d76fd6652") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:28:03.470960 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:03.470923 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkbtt\" (UniqueName: \"kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt\") pod \"network-check-target-28tgs\" (UID: \"46dc289d-2577-400d-84d5-8411d4225d46\") " pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:03.471126 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:03.471047 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:28:03.471126 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:03.471063 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:28:03.471126 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:03.471072 2561 projected.go:196] Error preparing data for projected volume kube-api-access-vkbtt for pod openshift-network-diagnostics/network-check-target-28tgs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:28:03.471126 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:03.471122 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt podName:46dc289d-2577-400d-84d5-8411d4225d46 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:19.471108825 +0000 UTC m=+34.338784362 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vkbtt" (UniqueName: "kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt") pod "network-check-target-28tgs" (UID: "46dc289d-2577-400d-84d5-8411d4225d46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:28:03.647231 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:03.646944 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lg8j5_bc341ace-408b-47f4-8fbf-adaf13c4772f/dns-node-resolver/0.log" Jul 06 19:28:03.681474 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:03.681435 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:03.681474 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:03.681463 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:03.681703 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:03.681585 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:03.681761 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:03.681714 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:04.680873 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:04.680839 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:04.681308 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:04.680945 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:04.820162 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:04.820137 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rk8nh_766b167f-bf67-495e-86e7-1a06223c14ae/node-ca/0.log" Jul 06 19:28:05.681660 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:05.681618 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:05.682105 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:05.681755 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:05.682105 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:05.681809 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:05.682105 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:05.681924 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:06.681039 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:06.681005 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:06.681218 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:06.681129 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:07.680774 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:07.680737 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:07.681344 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:07.680792 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:07.681344 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:07.680873 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:07.681344 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:07.681011 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:08.681617 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:08.681588 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:08.682123 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:08.681716 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:09.003810 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.003770 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 19:28:09.597912 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.597543 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T19:28:09.003793203Z","UUID":"79605d4e-187a-4053-b452-bfeb2bf74464","Handler":null,"Name":"","Endpoint":""} Jul 06 19:28:09.599242 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.599217 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 19:28:09.599416 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.599263 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 19:28:09.680779 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.680745 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:09.681090 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:09.680867 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:09.681090 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.680922 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:09.681090 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:09.681022 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:09.863477 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.863386 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" event={"ID":"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7","Type":"ContainerStarted","Data":"a026d2cb0360c4cd6cdd6254363d6ac22375f466101650a08533e996db48e3d3"} Jul 06 19:28:09.866925 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.866894 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" event={"ID":"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7","Type":"ContainerStarted","Data":"f756861e7db4af84f56393421c6a39974eba26709dbf0075639148e6a63b22e9"} Jul 06 19:28:09.867060 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.866990 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" event={"ID":"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7","Type":"ContainerStarted","Data":"e95d6a5d2b67654251dd526ecc46b9e22e0b6b40e25873f2f9efcb8643203e95"} Jul 06 19:28:09.867060 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.867041 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" event={"ID":"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7","Type":"ContainerStarted","Data":"b98d02f2865dbf60b266b44da06a8c15e09963c4f60dc95f101d48edf53bea69"} Jul 06 19:28:09.867152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.867091 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" event={"ID":"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7","Type":"ContainerStarted","Data":"035d25f3f35483d903110519612e7e9e4078a15e88e2a48006c5ae9557056738"} Jul 06 19:28:09.867152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.867138 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" event={"ID":"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7","Type":"ContainerStarted","Data":"1e969d84087e0f6c1c0b7028ee3d17509b527998a8fa1fb16f1088ba6ba6f811"} Jul 06 19:28:09.867251 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.867184 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" event={"ID":"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7","Type":"ContainerStarted","Data":"d36e7c0908811e043f42a6e944ec4ed65f229884a2cccef73c7c31ff2587f226"} Jul 06 19:28:09.868635 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.868613 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lmp4g" event={"ID":"62bc027b-c11e-4a1b-80e3-68dc5d8f8441","Type":"ContainerStarted","Data":"53b0e324a9710e23f254efeebe7395edee311ffbc8b6ea45b7814244231ae9b3"} Jul 06 19:28:09.894086 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:09.894013 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lmp4g" podStartSLOduration=3.92922891 podStartE2EDuration="23.89398703s" totalImagesPullingTime="19.96475812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:09.893339927 +0000 UTC m=+24.761015483" watchObservedRunningTime="2026-07-06 19:28:09.89398703 +0000 UTC m=+24.761662591" Jul 06 19:28:10.680694 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:10.680666 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:10.680893 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:10.680789 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:10.872330 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:10.872276 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" event={"ID":"cd5fa48a-ec49-459e-8748-6ac2dc6c49e7","Type":"ContainerStarted","Data":"bec231a4e75b6b230c35a771b24d219be0454680b8699dba39109a335cfca481"} Jul 06 19:28:10.894688 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:10.894068 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rd6b7" podStartSLOduration=3.1565934 podStartE2EDuration="23.894041482s" totalImagesPullingTime="20.737448082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:27:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:10.893130674 +0000 UTC m=+25.760806233" watchObservedRunningTime="2026-07-06 19:28:10.894041482 +0000 UTC m=+25.761717038" Jul 06 19:28:11.681695 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:11.681663 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:11.681839 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:11.681663 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:11.681839 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:11.681817 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:11.681946 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:11.681899 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:11.878216 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:11.878129 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" event={"ID":"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7","Type":"ContainerStarted","Data":"c4577fcc824a8f5cdb915b77a6d1c3495e9ae4a68aa31e77f3cd4b133436a4e0"} Jul 06 19:28:12.681612 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:12.681579 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:12.681820 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:12.681696 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:13.681337 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:13.681286 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:13.681929 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:13.681466 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:13.681929 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:13.681509 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:13.681929 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:13.681600 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:14.681518 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:14.681345 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:14.681826 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:14.681597 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:14.889275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:14.889249 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" event={"ID":"3d23fd58-2dcd-4259-ad3f-a96d811ab0b7","Type":"ContainerStarted","Data":"98c535635923b3b037717ab65f47823ab50b88d038105b4af9cfedfae55e8dd3"} Jul 06 19:28:14.889643 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:14.889617 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:28:14.889813 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:14.889701 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:28:14.893379 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:14.890084 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:28:14.909120 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:14.909087 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:28:14.910449 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:14.910427 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:28:14.928933 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:14.928787 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" podStartSLOduration=8.958843326 podStartE2EDuration="28.928768642s" totalImagesPullingTime="19.969925316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:14.927868085 +0000 UTC m=+29.795543646" watchObservedRunningTime="2026-07-06 19:28:14.928768642 +0000 UTC m=+29.796444199" Jul 06 19:28:15.682121 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:15.682092 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:15.682542 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:15.682176 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:15.682542 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:15.682204 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:15.682542 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:15.682249 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:15.893445 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:15.893410 2561 generic.go:356] "Generic (PLEG): container finished" podID="da99b45a-4a58-47d1-a98f-a05a8bb9801c" containerID="667827ca8a0a69222e4ad9fa95e4ce47c05e92cdcb0fe657cac8e3638969e459" exitCode=0 Jul 06 19:28:15.893636 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:15.893485 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kdjjh" event={"ID":"da99b45a-4a58-47d1-a98f-a05a8bb9801c","Type":"ContainerDied","Data":"667827ca8a0a69222e4ad9fa95e4ce47c05e92cdcb0fe657cac8e3638969e459"} Jul 06 19:28:16.681351 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:16.681313 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:16.681502 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:16.681421 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:17.681043 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:17.681009 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:17.681415 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:17.681009 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:17.681415 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:17.681123 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:17.681415 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:17.681234 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:17.899560 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:17.899530 2561 generic.go:356] "Generic (PLEG): container finished" podID="da99b45a-4a58-47d1-a98f-a05a8bb9801c" containerID="e0a24cf3dcf69911a77a0c6eb103f53948f37baeeeb19bdc34b862ed70d1f834" exitCode=0 Jul 06 19:28:17.899709 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:17.899567 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kdjjh" event={"ID":"da99b45a-4a58-47d1-a98f-a05a8bb9801c","Type":"ContainerDied","Data":"e0a24cf3dcf69911a77a0c6eb103f53948f37baeeeb19bdc34b862ed70d1f834"} Jul 06 19:28:18.681101 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:18.681052 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:18.681524 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:18.681173 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:18.903957 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:18.903868 2561 generic.go:356] "Generic (PLEG): container finished" podID="da99b45a-4a58-47d1-a98f-a05a8bb9801c" containerID="5bef102bc89354621c6638df6ef894bbe2ea340106221342ba99c593041c0c52" exitCode=0 Jul 06 19:28:18.903957 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:18.903929 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kdjjh" event={"ID":"da99b45a-4a58-47d1-a98f-a05a8bb9801c","Type":"ContainerDied","Data":"5bef102bc89354621c6638df6ef894bbe2ea340106221342ba99c593041c0c52"} Jul 06 19:28:19.286264 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:19.286229 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs\") pod \"network-metrics-daemon-26crl\" (UID: \"477cd95b-0bfb-4470-8226-9ee5ed229f99\") " pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:19.286427 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:19.286346 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:28:19.286427 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:19.286396 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs podName:477cd95b-0bfb-4470-8226-9ee5ed229f99 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:51.286381617 +0000 UTC m=+66.154057151 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs") pod "network-metrics-daemon-26crl" (UID: "477cd95b-0bfb-4470-8226-9ee5ed229f99") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:28:19.387411 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:19.387379 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:19.387589 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:19.387523 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:28:19.387589 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:19.387581 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret podName:4d150905-fd23-4a84-9a45-729d76fd6652 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:51.38756312 +0000 UTC m=+66.255238671 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret") pod "global-pull-secret-syncer-ddpss" (UID: "4d150905-fd23-4a84-9a45-729d76fd6652") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:28:19.488090 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:19.488057 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkbtt\" (UniqueName: \"kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt\") pod \"network-check-target-28tgs\" (UID: \"46dc289d-2577-400d-84d5-8411d4225d46\") " pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:19.488234 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:19.488189 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:28:19.488234 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:19.488210 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:28:19.488234 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:19.488223 2561 projected.go:196] Error preparing data for projected volume kube-api-access-vkbtt for pod openshift-network-diagnostics/network-check-target-28tgs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:28:19.488400 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:19.488283 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt podName:46dc289d-2577-400d-84d5-8411d4225d46 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:51.48826532 +0000 UTC m=+66.355940858 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vkbtt" (UniqueName: "kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt") pod "network-check-target-28tgs" (UID: "46dc289d-2577-400d-84d5-8411d4225d46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:28:19.681338 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:19.681307 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:19.681758 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:19.681450 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:19.681758 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:19.681515 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:19.681758 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:19.681627 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:20.680780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:20.680750 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:20.681011 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:20.680847 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:21.681638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:21.681429 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:21.682105 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:21.681488 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:21.682105 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:21.681755 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:21.682105 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:21.681906 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:22.680996 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:22.680962 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:22.681176 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:22.681072 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:23.680901 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:23.680872 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:23.680901 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:23.680900 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:23.681283 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:23.680978 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:23.681283 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:23.681047 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:24.680620 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:24.680586 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:24.680768 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:24.680687 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:24.922945 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:24.922916 2561 generic.go:356] "Generic (PLEG): container finished" podID="da99b45a-4a58-47d1-a98f-a05a8bb9801c" containerID="02b82d3ca0072a55f9e2d28cae3d4bf4a581fcfc7b353bdc447a8d8c8b54c26b" exitCode=0 Jul 06 19:28:24.923309 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:24.922965 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kdjjh" event={"ID":"da99b45a-4a58-47d1-a98f-a05a8bb9801c","Type":"ContainerDied","Data":"02b82d3ca0072a55f9e2d28cae3d4bf4a581fcfc7b353bdc447a8d8c8b54c26b"} Jul 06 19:28:25.681501 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:25.681471 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:25.681659 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:25.681576 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:25.681659 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:25.681630 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:25.681737 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:25.681715 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:25.927962 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:25.927929 2561 generic.go:356] "Generic (PLEG): container finished" podID="da99b45a-4a58-47d1-a98f-a05a8bb9801c" containerID="30570863a7a658e59d4bc4598f52179e5cf357c6d53c590d5420919b052668a9" exitCode=0 Jul 06 19:28:25.928444 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:25.927970 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kdjjh" event={"ID":"da99b45a-4a58-47d1-a98f-a05a8bb9801c","Type":"ContainerDied","Data":"30570863a7a658e59d4bc4598f52179e5cf357c6d53c590d5420919b052668a9"} Jul 06 19:28:26.680988 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:26.680957 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:26.681137 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:26.681052 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:26.933478 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:26.933405 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kdjjh" event={"ID":"da99b45a-4a58-47d1-a98f-a05a8bb9801c","Type":"ContainerStarted","Data":"7f64cd0b6e60785a800cc071b642c189e6739898fdcf67a46ee23ffebd77c4da"} Jul 06 19:28:26.959129 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:26.959084 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kdjjh" podStartSLOduration=9.428377014 podStartE2EDuration="40.959068716s" totalImagesPullingTime="31.530691702s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:27:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:26.957972014 +0000 UTC m=+41.825647569" watchObservedRunningTime="2026-07-06 19:28:26.959068716 +0000 UTC m=+41.826744271" Jul 06 19:28:27.681510 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:27.681473 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:27.681696 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:27.681481 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:27.681696 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:27.681605 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:27.681696 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:27.681680 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:27.943985 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:27.943882 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ddpss"] Jul 06 19:28:27.943985 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:27.943975 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:27.944744 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:27.944068 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:27.946508 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:27.946485 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-28tgs"] Jul 06 19:28:27.946632 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:27.946566 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:27.946681 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:27.946655 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:27.958821 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:27.958797 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-26crl"] Jul 06 19:28:27.958925 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:27.958866 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:27.958964 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:27.958944 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:29.681580 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:29.681545 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:29.681580 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:29.681545 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:29.682044 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:29.681699 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-28tgs" podUID="46dc289d-2577-400d-84d5-8411d4225d46" Jul 06 19:28:29.682044 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:29.681745 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:29.682044 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:29.681812 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddpss" podUID="4d150905-fd23-4a84-9a45-729d76fd6652" Jul 06 19:28:29.682044 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:29.681886 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-26crl" podUID="477cd95b-0bfb-4470-8226-9ee5ed229f99" Jul 06 19:28:31.419752 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.419588 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-33.ec2.internal" event="NodeReady" Jul 06 19:28:31.420089 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.419851 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 19:28:31.468882 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.468851 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-q9dm7"] Jul 06 19:28:31.491635 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.491604 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-84nwx"] Jul 06 19:28:31.491764 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.491726 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:31.494871 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.494846 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 19:28:31.494871 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.494846 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6kw96\"" Jul 06 19:28:31.495050 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.494888 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 19:28:31.514809 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.514785 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q9dm7"] Jul 06 19:28:31.514914 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.514829 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-84nwx" Jul 06 19:28:31.514914 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.514839 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-84nwx"] Jul 06 19:28:31.514914 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.514863 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-x66xp"] Jul 06 19:28:31.518383 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.518358 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 19:28:31.518383 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.518375 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qg2nr\"" Jul 06 19:28:31.518557 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.518394 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 19:28:31.518557 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.518412 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 19:28:31.538874 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.538848 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x66xp"] Jul 06 19:28:31.539001 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.538956 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.543133 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.543115 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 19:28:31.543252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.543239 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nmc6g\"" Jul 06 19:28:31.543689 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.543614 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 19:28:31.543689 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.543614 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 19:28:31.543689 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.543653 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 19:28:31.572054 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.571953 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9a4277f-542c-4748-a202-56ad0cf65ebd-cert\") pod \"ingress-canary-84nwx\" (UID: \"a9a4277f-542c-4748-a202-56ad0cf65ebd\") " pod="openshift-ingress-canary/ingress-canary-84nwx" Jul 06 19:28:31.572054 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.571985 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86lhp\" (UniqueName: \"kubernetes.io/projected/825798dc-9fa7-44a5-8cd4-559fe23a6b94-kube-api-access-86lhp\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.572054 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.572004 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flcfs\" (UniqueName: \"kubernetes.io/projected/a9a4277f-542c-4748-a202-56ad0cf65ebd-kube-api-access-flcfs\") pod \"ingress-canary-84nwx\" (UID: \"a9a4277f-542c-4748-a202-56ad0cf65ebd\") " pod="openshift-ingress-canary/ingress-canary-84nwx" Jul 06 19:28:31.572054 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.572026 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/825798dc-9fa7-44a5-8cd4-559fe23a6b94-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.572332 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.572102 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8233bc1c-af16-451c-bd31-29c9b3695124-config-volume\") pod \"dns-default-q9dm7\" (UID: \"8233bc1c-af16-451c-bd31-29c9b3695124\") " pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:31.572332 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.572135 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5wpd\" (UniqueName: \"kubernetes.io/projected/8233bc1c-af16-451c-bd31-29c9b3695124-kube-api-access-r5wpd\") pod \"dns-default-q9dm7\" (UID: \"8233bc1c-af16-451c-bd31-29c9b3695124\") " pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:31.572332 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.572195 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8233bc1c-af16-451c-bd31-29c9b3695124-metrics-tls\") pod \"dns-default-q9dm7\" (UID: \"8233bc1c-af16-451c-bd31-29c9b3695124\") " pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:31.572332 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.572219 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/825798dc-9fa7-44a5-8cd4-559fe23a6b94-data-volume\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.572332 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.572241 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/825798dc-9fa7-44a5-8cd4-559fe23a6b94-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.572332 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.572263 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8233bc1c-af16-451c-bd31-29c9b3695124-tmp-dir\") pod \"dns-default-q9dm7\" (UID: \"8233bc1c-af16-451c-bd31-29c9b3695124\") " pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:31.572332 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.572283 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/825798dc-9fa7-44a5-8cd4-559fe23a6b94-crio-socket\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.673465 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.673431 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8233bc1c-af16-451c-bd31-29c9b3695124-config-volume\") pod \"dns-default-q9dm7\" (UID: \"8233bc1c-af16-451c-bd31-29c9b3695124\") " pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:31.673465 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.673466 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5wpd\" (UniqueName: \"kubernetes.io/projected/8233bc1c-af16-451c-bd31-29c9b3695124-kube-api-access-r5wpd\") pod \"dns-default-q9dm7\" (UID: \"8233bc1c-af16-451c-bd31-29c9b3695124\") " pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:31.673709 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.673493 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8233bc1c-af16-451c-bd31-29c9b3695124-metrics-tls\") pod \"dns-default-q9dm7\" (UID: \"8233bc1c-af16-451c-bd31-29c9b3695124\") " pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:31.673709 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.673513 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/825798dc-9fa7-44a5-8cd4-559fe23a6b94-data-volume\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.673709 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.673529 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/825798dc-9fa7-44a5-8cd4-559fe23a6b94-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.673709 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.673551 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8233bc1c-af16-451c-bd31-29c9b3695124-tmp-dir\") pod \"dns-default-q9dm7\" (UID: \"8233bc1c-af16-451c-bd31-29c9b3695124\") " pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:31.673709 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.673582 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/825798dc-9fa7-44a5-8cd4-559fe23a6b94-crio-socket\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.673709 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.673629 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9a4277f-542c-4748-a202-56ad0cf65ebd-cert\") pod \"ingress-canary-84nwx\" (UID: \"a9a4277f-542c-4748-a202-56ad0cf65ebd\") " pod="openshift-ingress-canary/ingress-canary-84nwx" Jul 06 19:28:31.673709 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.673654 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86lhp\" (UniqueName: \"kubernetes.io/projected/825798dc-9fa7-44a5-8cd4-559fe23a6b94-kube-api-access-86lhp\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.673709 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.673679 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flcfs\" (UniqueName: \"kubernetes.io/projected/a9a4277f-542c-4748-a202-56ad0cf65ebd-kube-api-access-flcfs\") pod \"ingress-canary-84nwx\" (UID: \"a9a4277f-542c-4748-a202-56ad0cf65ebd\") " pod="openshift-ingress-canary/ingress-canary-84nwx" Jul 06 19:28:31.674085 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.673719 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/825798dc-9fa7-44a5-8cd4-559fe23a6b94-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.674085 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.673907 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/825798dc-9fa7-44a5-8cd4-559fe23a6b94-data-volume\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.674085 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.674008 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/825798dc-9fa7-44a5-8cd4-559fe23a6b94-crio-socket\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.674085 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.674045 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8233bc1c-af16-451c-bd31-29c9b3695124-tmp-dir\") pod \"dns-default-q9dm7\" (UID: \"8233bc1c-af16-451c-bd31-29c9b3695124\") " pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:31.674269 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.674188 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8233bc1c-af16-451c-bd31-29c9b3695124-config-volume\") pod \"dns-default-q9dm7\" (UID: \"8233bc1c-af16-451c-bd31-29c9b3695124\") " pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:31.674614 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.674594 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/825798dc-9fa7-44a5-8cd4-559fe23a6b94-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.677650 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.677622 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8233bc1c-af16-451c-bd31-29c9b3695124-metrics-tls\") pod \"dns-default-q9dm7\" (UID: \"8233bc1c-af16-451c-bd31-29c9b3695124\") " pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:31.677768 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.677686 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/825798dc-9fa7-44a5-8cd4-559fe23a6b94-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.677768 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.677741 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9a4277f-542c-4748-a202-56ad0cf65ebd-cert\") pod \"ingress-canary-84nwx\" (UID: \"a9a4277f-542c-4748-a202-56ad0cf65ebd\") " pod="openshift-ingress-canary/ingress-canary-84nwx" Jul 06 19:28:31.680796 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.680772 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:31.680892 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.680804 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:31.680953 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.680772 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:31.682417 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.682392 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86lhp\" (UniqueName: \"kubernetes.io/projected/825798dc-9fa7-44a5-8cd4-559fe23a6b94-kube-api-access-86lhp\") pod \"insights-runtime-extractor-x66xp\" (UID: \"825798dc-9fa7-44a5-8cd4-559fe23a6b94\") " pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:31.682486 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.682440 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flcfs\" (UniqueName: \"kubernetes.io/projected/a9a4277f-542c-4748-a202-56ad0cf65ebd-kube-api-access-flcfs\") pod \"ingress-canary-84nwx\" (UID: \"a9a4277f-542c-4748-a202-56ad0cf65ebd\") " pod="openshift-ingress-canary/ingress-canary-84nwx" Jul 06 19:28:31.682486 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.682472 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5wpd\" (UniqueName: \"kubernetes.io/projected/8233bc1c-af16-451c-bd31-29c9b3695124-kube-api-access-r5wpd\") pod \"dns-default-q9dm7\" (UID: \"8233bc1c-af16-451c-bd31-29c9b3695124\") " pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:31.686503 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.684736 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:28:31.686503 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.685029 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mjmjf\"" Jul 06 19:28:31.686503 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.685244 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:28:31.686503 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.685514 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:28:31.686503 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.685717 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vdwjn\"" Jul 06 19:28:31.686503 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.685915 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:28:31.801338 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.801277 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:31.823845 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.823805 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-84nwx" Jul 06 19:28:31.847153 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:31.847126 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x66xp" Jul 06 19:28:32.008724 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:32.008689 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-84nwx"] Jul 06 19:28:32.013122 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:32.013097 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9a4277f_542c_4748_a202_56ad0cf65ebd.slice/crio-071772ac772be1e0bcc542be3610e32990473839ca39c5bb58f1eea20125fe92 WatchSource:0}: Error finding container 071772ac772be1e0bcc542be3610e32990473839ca39c5bb58f1eea20125fe92: Status 404 returned error can't find the container with id 071772ac772be1e0bcc542be3610e32990473839ca39c5bb58f1eea20125fe92 Jul 06 19:28:32.035822 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:32.035797 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x66xp"] Jul 06 19:28:32.038930 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:32.038905 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod825798dc_9fa7_44a5_8cd4_559fe23a6b94.slice/crio-4c366c6dccd98dff060230c0f7f33163001aa7b9a5b96340e29ba0d0ce1ad302 WatchSource:0}: Error finding container 4c366c6dccd98dff060230c0f7f33163001aa7b9a5b96340e29ba0d0ce1ad302: Status 404 returned error can't find the container with id 4c366c6dccd98dff060230c0f7f33163001aa7b9a5b96340e29ba0d0ce1ad302 Jul 06 19:28:32.042239 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:32.042221 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q9dm7"] Jul 06 19:28:32.044553 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:32.044529 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8233bc1c_af16_451c_bd31_29c9b3695124.slice/crio-a7830aa48b9a98bc4f353ad2e83800dc72e077b676e57ee06381438a9f453dfa WatchSource:0}: Error finding container a7830aa48b9a98bc4f353ad2e83800dc72e077b676e57ee06381438a9f453dfa: Status 404 returned error can't find the container with id a7830aa48b9a98bc4f353ad2e83800dc72e077b676e57ee06381438a9f453dfa Jul 06 19:28:32.956543 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:32.956504 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-84nwx" event={"ID":"a9a4277f-542c-4748-a202-56ad0cf65ebd","Type":"ContainerStarted","Data":"071772ac772be1e0bcc542be3610e32990473839ca39c5bb58f1eea20125fe92"} Jul 06 19:28:32.959625 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:32.959592 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q9dm7" event={"ID":"8233bc1c-af16-451c-bd31-29c9b3695124","Type":"ContainerStarted","Data":"a7830aa48b9a98bc4f353ad2e83800dc72e077b676e57ee06381438a9f453dfa"} Jul 06 19:28:32.961136 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:32.961109 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x66xp" event={"ID":"825798dc-9fa7-44a5-8cd4-559fe23a6b94","Type":"ContainerStarted","Data":"4c366c6dccd98dff060230c0f7f33163001aa7b9a5b96340e29ba0d0ce1ad302"} Jul 06 19:28:35.869568 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.869536 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c646d46d5-bfp57"] Jul 06 19:28:35.888389 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.888360 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c646d46d5-bfp57"] Jul 06 19:28:35.888542 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.888480 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:35.892156 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.892128 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 19:28:35.892312 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.892247 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 19:28:35.892640 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.892469 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 19:28:35.893620 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.893366 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cj9rh\"" Jul 06 19:28:35.893620 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.893597 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 19:28:35.896517 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.893812 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 19:28:35.896517 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.894027 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 19:28:35.896517 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.894903 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 19:28:35.907391 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.907370 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-console-config\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:35.907491 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.907463 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-service-ca\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:35.907546 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.907495 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-oauth-serving-cert\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:35.907546 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.907542 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c819dedf-7c04-4003-8555-fdb5e5857f22-console-oauth-config\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:35.907628 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.907578 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6v7l\" (UniqueName: \"kubernetes.io/projected/c819dedf-7c04-4003-8555-fdb5e5857f22-kube-api-access-b6v7l\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:35.907628 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:35.907602 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c819dedf-7c04-4003-8555-fdb5e5857f22-console-serving-cert\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:36.007997 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:36.007953 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6v7l\" (UniqueName: \"kubernetes.io/projected/c819dedf-7c04-4003-8555-fdb5e5857f22-kube-api-access-b6v7l\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:36.008180 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:36.008013 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c819dedf-7c04-4003-8555-fdb5e5857f22-console-serving-cert\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:36.008180 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:36.008047 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-console-config\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:36.008180 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:36.008108 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-service-ca\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:36.008180 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:36.008132 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-oauth-serving-cert\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:36.008421 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:36.008348 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c819dedf-7c04-4003-8555-fdb5e5857f22-console-oauth-config\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:36.008906 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:36.008886 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-oauth-serving-cert\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:36.008906 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:36.008895 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-service-ca\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:36.009067 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:36.008926 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-console-config\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:36.011864 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:36.011810 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c819dedf-7c04-4003-8555-fdb5e5857f22-console-oauth-config\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:36.011864 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:36.011839 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c819dedf-7c04-4003-8555-fdb5e5857f22-console-serving-cert\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:36.017862 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:36.017832 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6v7l\" (UniqueName: \"kubernetes.io/projected/c819dedf-7c04-4003-8555-fdb5e5857f22-kube-api-access-b6v7l\") pod \"console-7c646d46d5-bfp57\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:36.201159 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:36.201116 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:37.150124 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.150090 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp"] Jul 06 19:28:37.165669 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.165643 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" Jul 06 19:28:37.169605 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.169588 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 19:28:37.169762 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.169645 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 19:28:37.171135 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.171116 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 19:28:37.171242 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.171225 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7g8bw\"" Jul 06 19:28:37.171304 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.171236 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 19:28:37.171349 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.171322 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 19:28:37.176581 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.176559 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp"] Jul 06 19:28:37.216898 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.216865 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec9d6541-fd28-44b5-9746-4f8ca3cde847-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-29kfp\" (UID: \"ec9d6541-fd28-44b5-9746-4f8ca3cde847\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" Jul 06 19:28:37.217048 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.216906 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf7zr\" (UniqueName: \"kubernetes.io/projected/ec9d6541-fd28-44b5-9746-4f8ca3cde847-kube-api-access-jf7zr\") pod \"prometheus-operator-567f7b8b7c-29kfp\" (UID: \"ec9d6541-fd28-44b5-9746-4f8ca3cde847\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" Jul 06 19:28:37.217048 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.216976 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec9d6541-fd28-44b5-9746-4f8ca3cde847-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-29kfp\" (UID: \"ec9d6541-fd28-44b5-9746-4f8ca3cde847\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" Jul 06 19:28:37.217048 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.217022 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec9d6541-fd28-44b5-9746-4f8ca3cde847-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-29kfp\" (UID: \"ec9d6541-fd28-44b5-9746-4f8ca3cde847\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" Jul 06 19:28:37.317798 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.317762 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf7zr\" (UniqueName: \"kubernetes.io/projected/ec9d6541-fd28-44b5-9746-4f8ca3cde847-kube-api-access-jf7zr\") pod \"prometheus-operator-567f7b8b7c-29kfp\" (UID: \"ec9d6541-fd28-44b5-9746-4f8ca3cde847\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" Jul 06 19:28:37.317967 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.317826 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec9d6541-fd28-44b5-9746-4f8ca3cde847-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-29kfp\" (UID: \"ec9d6541-fd28-44b5-9746-4f8ca3cde847\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" Jul 06 19:28:37.317967 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.317855 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec9d6541-fd28-44b5-9746-4f8ca3cde847-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-29kfp\" (UID: \"ec9d6541-fd28-44b5-9746-4f8ca3cde847\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" Jul 06 19:28:37.318107 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.318071 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec9d6541-fd28-44b5-9746-4f8ca3cde847-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-29kfp\" (UID: \"ec9d6541-fd28-44b5-9746-4f8ca3cde847\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" Jul 06 19:28:37.318695 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.318673 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec9d6541-fd28-44b5-9746-4f8ca3cde847-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-29kfp\" (UID: \"ec9d6541-fd28-44b5-9746-4f8ca3cde847\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" Jul 06 19:28:37.320244 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.320222 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec9d6541-fd28-44b5-9746-4f8ca3cde847-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-29kfp\" (UID: \"ec9d6541-fd28-44b5-9746-4f8ca3cde847\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" Jul 06 19:28:37.320440 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.320418 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec9d6541-fd28-44b5-9746-4f8ca3cde847-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-29kfp\" (UID: \"ec9d6541-fd28-44b5-9746-4f8ca3cde847\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" Jul 06 19:28:37.327593 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.327567 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf7zr\" (UniqueName: \"kubernetes.io/projected/ec9d6541-fd28-44b5-9746-4f8ca3cde847-kube-api-access-jf7zr\") pod \"prometheus-operator-567f7b8b7c-29kfp\" (UID: \"ec9d6541-fd28-44b5-9746-4f8ca3cde847\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" Jul 06 19:28:37.475015 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.474966 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" Jul 06 19:28:37.899581 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.899558 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp"] Jul 06 19:28:37.913920 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.913888 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c646d46d5-bfp57"] Jul 06 19:28:37.977193 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.977156 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-84nwx" event={"ID":"a9a4277f-542c-4748-a202-56ad0cf65ebd","Type":"ContainerStarted","Data":"ac9587423ab147baffaeaecacaef5e3525e99ace5b852098c5b5f8368ffb4d0e"} Jul 06 19:28:37.978782 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.978757 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q9dm7" event={"ID":"8233bc1c-af16-451c-bd31-29c9b3695124","Type":"ContainerStarted","Data":"157744ba0ea32447716140c400c7a9efca9c0619b3beee88a881b7a0d7dd1ddf"} Jul 06 19:28:37.997004 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:37.996918 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-84nwx" podStartSLOduration=1.265183956 podStartE2EDuration="6.99690359s" totalImagesPullingTime="5.731719634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:37.995984635 +0000 UTC m=+52.863660190" watchObservedRunningTime="2026-07-06 19:28:37.99690359 +0000 UTC m=+52.864579147" Jul 06 19:28:38.020493 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:38.020454 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec9d6541_fd28_44b5_9746_4f8ca3cde847.slice/crio-94ab77ee47e9797dee54600f4641ee71793fde246cb6f77711dac2c62f8d033a WatchSource:0}: Error finding container 94ab77ee47e9797dee54600f4641ee71793fde246cb6f77711dac2c62f8d033a: Status 404 returned error can't find the container with id 94ab77ee47e9797dee54600f4641ee71793fde246cb6f77711dac2c62f8d033a Jul 06 19:28:38.021415 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:38.021277 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc819dedf_7c04_4003_8555_fdb5e5857f22.slice/crio-1b73c21d3cb7b6cbed34f31b2f6403a6997c647b1f884a0fceee76e4f152f81c WatchSource:0}: Error finding container 1b73c21d3cb7b6cbed34f31b2f6403a6997c647b1f884a0fceee76e4f152f81c: Status 404 returned error can't find the container with id 1b73c21d3cb7b6cbed34f31b2f6403a6997c647b1f884a0fceee76e4f152f81c Jul 06 19:28:38.984069 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:38.984029 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q9dm7" event={"ID":"8233bc1c-af16-451c-bd31-29c9b3695124","Type":"ContainerStarted","Data":"419a7dd374f1a4392f63cde179b20b5a8996e52e3ca5c8ab60f075fb52737f91"} Jul 06 19:28:38.984693 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:38.984136 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:38.985886 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:38.985813 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x66xp" event={"ID":"825798dc-9fa7-44a5-8cd4-559fe23a6b94","Type":"ContainerStarted","Data":"6abf1fcebac4709d11b9708c7baf1962fe4a02caa8206541307fc9ca154541ea"} Jul 06 19:28:38.987164 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:38.987136 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" event={"ID":"ec9d6541-fd28-44b5-9746-4f8ca3cde847","Type":"ContainerStarted","Data":"94ab77ee47e9797dee54600f4641ee71793fde246cb6f77711dac2c62f8d033a"} Jul 06 19:28:38.988574 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:38.988544 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c646d46d5-bfp57" event={"ID":"c819dedf-7c04-4003-8555-fdb5e5857f22","Type":"ContainerStarted","Data":"1b73c21d3cb7b6cbed34f31b2f6403a6997c647b1f884a0fceee76e4f152f81c"} Jul 06 19:28:39.008961 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:39.008902 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-q9dm7" podStartSLOduration=2.314406765 podStartE2EDuration="8.008883502s" totalImagesPullingTime="5.694476737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:39.007560105 +0000 UTC m=+53.875235662" watchObservedRunningTime="2026-07-06 19:28:39.008883502 +0000 UTC m=+53.876559037" Jul 06 19:28:39.995016 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:39.994981 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x66xp" event={"ID":"825798dc-9fa7-44a5-8cd4-559fe23a6b94","Type":"ContainerStarted","Data":"e7252a4e7c1d1d2507447f1f3d12c978398f9551c63a6dfe407e785f742eb1c0"} Jul 06 19:28:40.000632 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:39.999816 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" event={"ID":"ec9d6541-fd28-44b5-9746-4f8ca3cde847","Type":"ContainerStarted","Data":"7fad7e583e17b21c36908e3fbef6b4cd8cfcce251319f65e44aeaa2eb96259a4"} Jul 06 19:28:40.000632 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:39.999878 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" event={"ID":"ec9d6541-fd28-44b5-9746-4f8ca3cde847","Type":"ContainerStarted","Data":"7fd12e90d375bedc9463e0564b0ccdb73f2c9162e96c324b54d822875531e33e"} Jul 06 19:28:40.029718 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:40.029627 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-29kfp" podStartSLOduration=1.3204568239999999 podStartE2EDuration="3.029602664s" totalImagesPullingTime="1.70914584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:40.028801901 +0000 UTC m=+54.896477460" watchObservedRunningTime="2026-07-06 19:28:40.029602664 +0000 UTC m=+54.897278219" Jul 06 19:28:41.574837 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.574768 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5"] Jul 06 19:28:41.577385 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.577365 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" Jul 06 19:28:41.583795 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.583770 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 19:28:41.583795 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.583786 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 19:28:41.585631 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.585613 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bp44x\"" Jul 06 19:28:41.611831 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.611747 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5"] Jul 06 19:28:41.612505 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.612481 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b"] Jul 06 19:28:41.614814 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.614792 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.618406 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.618387 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 19:28:41.618694 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.618675 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mrjcz\"" Jul 06 19:28:41.618815 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.618749 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 19:28:41.619467 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.619448 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 19:28:41.632396 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.632366 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-62b95"] Jul 06 19:28:41.634885 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.634511 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.640517 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.640442 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fxbq4\"" Jul 06 19:28:41.640638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.640526 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 19:28:41.640989 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.640950 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 19:28:41.641087 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.641029 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 19:28:41.648926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.648906 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b"] Jul 06 19:28:41.650785 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.650724 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66d06755-9fb1-4361-ae12-04286b7be77c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.650785 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.650765 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/66d06755-9fb1-4361-ae12-04286b7be77c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.651002 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.650792 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c302db2-20ca-4d3c-91ff-5ce9c6565adf-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-8gkq5\" (UID: \"3c302db2-20ca-4d3c-91ff-5ce9c6565adf\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" Jul 06 19:28:41.651002 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.650834 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcqpx\" (UniqueName: \"kubernetes.io/projected/3c302db2-20ca-4d3c-91ff-5ce9c6565adf-kube-api-access-wcqpx\") pod \"openshift-state-metrics-54cf96dfcf-8gkq5\" (UID: \"3c302db2-20ca-4d3c-91ff-5ce9c6565adf\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" Jul 06 19:28:41.651002 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.650857 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/66d06755-9fb1-4361-ae12-04286b7be77c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.651002 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.650928 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c302db2-20ca-4d3c-91ff-5ce9c6565adf-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-8gkq5\" (UID: \"3c302db2-20ca-4d3c-91ff-5ce9c6565adf\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" Jul 06 19:28:41.651002 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.650967 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/66d06755-9fb1-4361-ae12-04286b7be77c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.651245 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.651015 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrzr4\" (UniqueName: \"kubernetes.io/projected/66d06755-9fb1-4361-ae12-04286b7be77c-kube-api-access-nrzr4\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.651245 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.651114 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c302db2-20ca-4d3c-91ff-5ce9c6565adf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-8gkq5\" (UID: \"3c302db2-20ca-4d3c-91ff-5ce9c6565adf\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" Jul 06 19:28:41.651245 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.651164 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66d06755-9fb1-4361-ae12-04286b7be77c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.751565 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.751523 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c302db2-20ca-4d3c-91ff-5ce9c6565adf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-8gkq5\" (UID: \"3c302db2-20ca-4d3c-91ff-5ce9c6565adf\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" Jul 06 19:28:41.751754 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.751568 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66d06755-9fb1-4361-ae12-04286b7be77c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.751754 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.751608 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66d06755-9fb1-4361-ae12-04286b7be77c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.751754 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.751638 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/66d06755-9fb1-4361-ae12-04286b7be77c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.751754 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.751671 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b97f973-3964-4944-af29-163662153573-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.751754 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.751697 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c302db2-20ca-4d3c-91ff-5ce9c6565adf-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-8gkq5\" (UID: \"3c302db2-20ca-4d3c-91ff-5ce9c6565adf\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" Jul 06 19:28:41.751754 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.751722 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcqpx\" (UniqueName: \"kubernetes.io/projected/3c302db2-20ca-4d3c-91ff-5ce9c6565adf-kube-api-access-wcqpx\") pod \"openshift-state-metrics-54cf96dfcf-8gkq5\" (UID: \"3c302db2-20ca-4d3c-91ff-5ce9c6565adf\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" Jul 06 19:28:41.751754 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.751744 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/66d06755-9fb1-4361-ae12-04286b7be77c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.752162 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.751788 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b97f973-3964-4944-af29-163662153573-node-exporter-wtmp\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.752162 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.751813 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b97f973-3964-4944-af29-163662153573-node-exporter-accelerators-collector-config\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.752162 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.751841 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c302db2-20ca-4d3c-91ff-5ce9c6565adf-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-8gkq5\" (UID: \"3c302db2-20ca-4d3c-91ff-5ce9c6565adf\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" Jul 06 19:28:41.752162 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.751870 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b97f973-3964-4944-af29-163662153573-metrics-client-ca\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.752162 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.751924 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/66d06755-9fb1-4361-ae12-04286b7be77c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.752162 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.751988 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b97f973-3964-4944-af29-163662153573-sys\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.752162 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.752029 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b97f973-3964-4944-af29-163662153573-node-exporter-tls\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.752162 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.752082 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrzr4\" (UniqueName: \"kubernetes.io/projected/66d06755-9fb1-4361-ae12-04286b7be77c-kube-api-access-nrzr4\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.752162 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.752154 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b97f973-3964-4944-af29-163662153573-node-exporter-textfile\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.752662 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.752199 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b97f973-3964-4944-af29-163662153573-root\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.752662 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.752230 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24k7l\" (UniqueName: \"kubernetes.io/projected/6b97f973-3964-4944-af29-163662153573-kube-api-access-24k7l\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.752662 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.752414 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66d06755-9fb1-4361-ae12-04286b7be77c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.753009 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.752971 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/66d06755-9fb1-4361-ae12-04286b7be77c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.753108 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.753023 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/66d06755-9fb1-4361-ae12-04286b7be77c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.753466 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.753436 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c302db2-20ca-4d3c-91ff-5ce9c6565adf-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-8gkq5\" (UID: \"3c302db2-20ca-4d3c-91ff-5ce9c6565adf\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" Jul 06 19:28:41.754799 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.754755 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c302db2-20ca-4d3c-91ff-5ce9c6565adf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-8gkq5\" (UID: \"3c302db2-20ca-4d3c-91ff-5ce9c6565adf\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" Jul 06 19:28:41.754972 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.754952 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66d06755-9fb1-4361-ae12-04286b7be77c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.755034 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.754972 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c302db2-20ca-4d3c-91ff-5ce9c6565adf-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-8gkq5\" (UID: \"3c302db2-20ca-4d3c-91ff-5ce9c6565adf\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" Jul 06 19:28:41.755184 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.755164 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/66d06755-9fb1-4361-ae12-04286b7be77c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.768066 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.768042 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcqpx\" (UniqueName: \"kubernetes.io/projected/3c302db2-20ca-4d3c-91ff-5ce9c6565adf-kube-api-access-wcqpx\") pod \"openshift-state-metrics-54cf96dfcf-8gkq5\" (UID: \"3c302db2-20ca-4d3c-91ff-5ce9c6565adf\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" Jul 06 19:28:41.768219 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.768087 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrzr4\" (UniqueName: \"kubernetes.io/projected/66d06755-9fb1-4361-ae12-04286b7be77c-kube-api-access-nrzr4\") pod \"kube-state-metrics-7c9dbb8b7d-q8v7b\" (UID: \"66d06755-9fb1-4361-ae12-04286b7be77c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:41.853134 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.853055 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b97f973-3964-4944-af29-163662153573-node-exporter-tls\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.853134 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.853093 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b97f973-3964-4944-af29-163662153573-node-exporter-textfile\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.853134 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.853131 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b97f973-3964-4944-af29-163662153573-root\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.853412 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.853158 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24k7l\" (UniqueName: \"kubernetes.io/projected/6b97f973-3964-4944-af29-163662153573-kube-api-access-24k7l\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.853412 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.853215 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b97f973-3964-4944-af29-163662153573-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.853412 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.853219 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b97f973-3964-4944-af29-163662153573-root\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.853412 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:41.853227 2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 19:28:41.853412 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.853259 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b97f973-3964-4944-af29-163662153573-node-exporter-wtmp\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.853412 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.853284 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b97f973-3964-4944-af29-163662153573-node-exporter-accelerators-collector-config\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.853412 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:41.853319 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b97f973-3964-4944-af29-163662153573-node-exporter-tls podName:6b97f973-3964-4944-af29-163662153573 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:42.353284093 +0000 UTC m=+57.220959628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6b97f973-3964-4944-af29-163662153573-node-exporter-tls") pod "node-exporter-62b95" (UID: "6b97f973-3964-4944-af29-163662153573") : secret "node-exporter-tls" not found Jul 06 19:28:41.853412 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.853348 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b97f973-3964-4944-af29-163662153573-metrics-client-ca\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.853412 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.853378 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b97f973-3964-4944-af29-163662153573-sys\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.853813 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.853444 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b97f973-3964-4944-af29-163662153573-sys\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.853813 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.853479 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b97f973-3964-4944-af29-163662153573-node-exporter-textfile\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.853813 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.853615 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b97f973-3964-4944-af29-163662153573-node-exporter-wtmp\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.853970 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.853838 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b97f973-3964-4944-af29-163662153573-node-exporter-accelerators-collector-config\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.854042 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.854004 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b97f973-3964-4944-af29-163662153573-metrics-client-ca\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.856122 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.856102 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b97f973-3964-4944-af29-163662153573-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.864588 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.864562 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24k7l\" (UniqueName: \"kubernetes.io/projected/6b97f973-3964-4944-af29-163662153573-kube-api-access-24k7l\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:41.888485 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.888443 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" Jul 06 19:28:41.926742 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:41.926701 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" Jul 06 19:28:42.011936 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.011641 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c646d46d5-bfp57" event={"ID":"c819dedf-7c04-4003-8555-fdb5e5857f22","Type":"ContainerStarted","Data":"cfa4992941b39b81802f6b016c23a862ad7732b431fd4685fb7d3708e1c2d6f0"} Jul 06 19:28:42.057009 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.056875 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c646d46d5-bfp57" podStartSLOduration=4.015399324 podStartE2EDuration="7.056851563s" totalImagesPullingTime="3.041452239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:42.035180854 +0000 UTC m=+56.902856410" watchObservedRunningTime="2026-07-06 19:28:42.056851563 +0000 UTC m=+56.924527120" Jul 06 19:28:42.057866 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.057787 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5"] Jul 06 19:28:42.060059 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:42.060031 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c302db2_20ca_4d3c_91ff_5ce9c6565adf.slice/crio-754c4d71fff3eebfb5a3bfc990448d130ada9153c18fcde569e6e38fe24063b0 WatchSource:0}: Error finding container 754c4d71fff3eebfb5a3bfc990448d130ada9153c18fcde569e6e38fe24063b0: Status 404 returned error can't find the container with id 754c4d71fff3eebfb5a3bfc990448d130ada9153c18fcde569e6e38fe24063b0 Jul 06 19:28:42.098103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.098072 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b"] Jul 06 19:28:42.102121 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:42.102089 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66d06755_9fb1_4361_ae12_04286b7be77c.slice/crio-d9031d2483e7df9c0d701b9e313eb2ecc4bd9bb9d58d03b1196603ff91573097 WatchSource:0}: Error finding container d9031d2483e7df9c0d701b9e313eb2ecc4bd9bb9d58d03b1196603ff91573097: Status 404 returned error can't find the container with id d9031d2483e7df9c0d701b9e313eb2ecc4bd9bb9d58d03b1196603ff91573097 Jul 06 19:28:42.358277 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.358201 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b97f973-3964-4944-af29-163662153573-node-exporter-tls\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:42.360968 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.360943 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b97f973-3964-4944-af29-163662153573-node-exporter-tls\") pod \"node-exporter-62b95\" (UID: \"6b97f973-3964-4944-af29-163662153573\") " pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:42.544965 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.544931 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-62b95" Jul 06 19:28:42.677489 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.677450 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:28:42.682620 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.682590 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.688355 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.688313 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 19:28:42.688506 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.688443 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 19:28:42.688666 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.688608 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 19:28:42.688799 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.688707 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 19:28:42.688799 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.688761 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 19:28:42.688799 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.688777 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 19:28:42.689118 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.689096 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 19:28:42.689203 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.689146 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s6dss\"" Jul 06 19:28:42.689203 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.689146 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 19:28:42.689203 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.689171 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 19:28:42.706975 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.706945 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:28:42.761187 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.761153 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.761391 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.761193 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.761391 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.761220 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.761391 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.761246 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdmtr\" (UniqueName: \"kubernetes.io/projected/a2095283-08a9-4fc7-b28f-46552ea28b04-kube-api-access-wdmtr\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.761391 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.761336 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.761610 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.761394 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.761610 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.761428 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2095283-08a9-4fc7-b28f-46552ea28b04-config-out\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.761610 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.761531 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2095283-08a9-4fc7-b28f-46552ea28b04-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.761610 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.761571 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.761610 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.761593 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.761806 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.761630 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-web-config\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.761806 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.761656 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.761806 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.761681 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-config-volume\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.862988 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.862951 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2095283-08a9-4fc7-b28f-46552ea28b04-config-out\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.863172 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.863047 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2095283-08a9-4fc7-b28f-46552ea28b04-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.863172 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.863080 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.863172 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.863098 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.863377 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.863271 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-web-config\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.863377 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.863332 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.863377 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.863366 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-config-volume\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.863531 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.863404 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.863531 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.863434 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.863531 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.863459 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.863531 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.863522 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdmtr\" (UniqueName: \"kubernetes.io/projected/a2095283-08a9-4fc7-b28f-46552ea28b04-kube-api-access-wdmtr\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.863735 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.863587 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.863735 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.863632 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.863872 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:42.863848 2561 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 06 19:28:42.863943 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:42.863929 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-main-tls podName:a2095283-08a9-4fc7-b28f-46552ea28b04 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:43.363906763 +0000 UTC m=+58.231582315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04") : secret "alertmanager-main-tls" not found Jul 06 19:28:42.864358 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:42.864274 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-trusted-ca-bundle podName:a2095283-08a9-4fc7-b28f-46552ea28b04 nodeName:}" failed. No retries permitted until 2026-07-06 19:28:43.364258476 +0000 UTC m=+58.231934025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04") : configmap references non-existent config key: ca-bundle.crt Jul 06 19:28:42.866549 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.866521 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2095283-08a9-4fc7-b28f-46552ea28b04-config-out\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.867070 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.866951 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.867070 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.867011 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-web-config\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.867585 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.867551 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2095283-08a9-4fc7-b28f-46552ea28b04-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.867999 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.867956 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.868098 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.868048 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.868161 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.868122 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.868408 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.868389 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.868492 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.868421 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.868597 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.868577 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-config-volume\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:42.875705 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:42.875677 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdmtr\" (UniqueName: \"kubernetes.io/projected/a2095283-08a9-4fc7-b28f-46552ea28b04-kube-api-access-wdmtr\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:43.017821 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:43.017743 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" event={"ID":"66d06755-9fb1-4361-ae12-04286b7be77c","Type":"ContainerStarted","Data":"d9031d2483e7df9c0d701b9e313eb2ecc4bd9bb9d58d03b1196603ff91573097"} Jul 06 19:28:43.019988 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:43.019952 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" event={"ID":"3c302db2-20ca-4d3c-91ff-5ce9c6565adf","Type":"ContainerStarted","Data":"240b9e9f9f31ef5c4192c39545b3ff32b0d1bcd990d827286645058b7262cf44"} Jul 06 19:28:43.020135 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:43.020018 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" event={"ID":"3c302db2-20ca-4d3c-91ff-5ce9c6565adf","Type":"ContainerStarted","Data":"3bdb8ede8380cc328ca25da0cae27499ea3a809c9a7feeddc1cb3c77421cd54b"} Jul 06 19:28:43.020135 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:43.020052 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" event={"ID":"3c302db2-20ca-4d3c-91ff-5ce9c6565adf","Type":"ContainerStarted","Data":"754c4d71fff3eebfb5a3bfc990448d130ada9153c18fcde569e6e38fe24063b0"} Jul 06 19:28:43.288944 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:43.288899 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b97f973_3964_4944_af29_163662153573.slice/crio-856f65a5ca1ee8a64553c802f52d504f66c5413a7bacdf46fb4ca6f0668aaf34 WatchSource:0}: Error finding container 856f65a5ca1ee8a64553c802f52d504f66c5413a7bacdf46fb4ca6f0668aaf34: Status 404 returned error can't find the container with id 856f65a5ca1ee8a64553c802f52d504f66c5413a7bacdf46fb4ca6f0668aaf34 Jul 06 19:28:43.368267 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:43.368235 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:43.368422 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:43.368308 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:43.369348 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:43.369276 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:43.371788 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:43.371741 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:43.594347 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:43.594172 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:28:44.026986 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.026944 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x66xp" event={"ID":"825798dc-9fa7-44a5-8cd4-559fe23a6b94","Type":"ContainerStarted","Data":"7c95128a4c3dfaa88849cf3a78c520e41ea6ff783be6c4ba2a27c65b2b305d73"} Jul 06 19:28:44.028409 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.028375 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-62b95" event={"ID":"6b97f973-3964-4944-af29-163662153573","Type":"ContainerStarted","Data":"856f65a5ca1ee8a64553c802f52d504f66c5413a7bacdf46fb4ca6f0668aaf34"} Jul 06 19:28:44.053080 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.053011 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-x66xp" podStartSLOduration=2.053398677 podStartE2EDuration="13.052988721s" totalImagesPullingTime="10.999590044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:28:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:44.050402164 +0000 UTC m=+58.918077721" watchObservedRunningTime="2026-07-06 19:28:44.052988721 +0000 UTC m=+58.920664280" Jul 06 19:28:44.508655 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.508621 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:28:44.514479 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:44.514441 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2095283_08a9_4fc7_b28f_46552ea28b04.slice/crio-0e01e04b34b778bf44ad9ccf72396075751a3856ee32dbd4c37118456e2e6057 WatchSource:0}: Error finding container 0e01e04b34b778bf44ad9ccf72396075751a3856ee32dbd4c37118456e2e6057: Status 404 returned error can't find the container with id 0e01e04b34b778bf44ad9ccf72396075751a3856ee32dbd4c37118456e2e6057 Jul 06 19:28:44.649767 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.649733 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5856747687-dl7fq"] Jul 06 19:28:44.652695 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.652670 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.658504 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.658480 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 19:28:44.658765 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.658480 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 19:28:44.658765 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.658557 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-nx7pq\"" Jul 06 19:28:44.658765 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.658588 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 19:28:44.658765 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.658489 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6h368rfb0l1j8\"" Jul 06 19:28:44.658991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.658869 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 19:28:44.659045 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.658997 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 19:28:44.667058 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.667017 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5856747687-dl7fq"] Jul 06 19:28:44.782567 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.782525 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g5vc\" (UniqueName: \"kubernetes.io/projected/4f267dd3-a02f-46f5-8041-b2c06becc979-kube-api-access-2g5vc\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.782567 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.782565 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.782815 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.782588 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-grpc-tls\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.782815 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.782661 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f267dd3-a02f-46f5-8041-b2c06becc979-metrics-client-ca\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.782815 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.782723 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-tls\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.782815 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.782757 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.782815 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.782779 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.783052 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.782832 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.884400 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.884271 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g5vc\" (UniqueName: \"kubernetes.io/projected/4f267dd3-a02f-46f5-8041-b2c06becc979-kube-api-access-2g5vc\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.884400 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.884355 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.884400 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.884386 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-grpc-tls\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.884672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.884416 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f267dd3-a02f-46f5-8041-b2c06becc979-metrics-client-ca\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.884672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.884504 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-tls\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.884672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.884576 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.884672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.884603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.884873 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.884679 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.887048 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.885135 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f267dd3-a02f-46f5-8041-b2c06becc979-metrics-client-ca\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.888222 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.888174 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.889019 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.888990 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-tls\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.889019 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.889012 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.889760 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.889495 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-grpc-tls\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.891631 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.891606 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.892629 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.892601 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4f267dd3-a02f-46f5-8041-b2c06becc979-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.897352 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.897283 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g5vc\" (UniqueName: \"kubernetes.io/projected/4f267dd3-a02f-46f5-8041-b2c06becc979-kube-api-access-2g5vc\") pod \"thanos-querier-5856747687-dl7fq\" (UID: \"4f267dd3-a02f-46f5-8041-b2c06becc979\") " pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:44.963477 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:44.963437 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:45.040277 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.040031 2561 generic.go:356] "Generic (PLEG): container finished" podID="6b97f973-3964-4944-af29-163662153573" containerID="8c5300a10a3fbf2c3e92b5e9ae04c7648bba22255d7b8fb0668802546581ca10" exitCode=0 Jul 06 19:28:45.040277 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.040161 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-62b95" event={"ID":"6b97f973-3964-4944-af29-163662153573","Type":"ContainerDied","Data":"8c5300a10a3fbf2c3e92b5e9ae04c7648bba22255d7b8fb0668802546581ca10"} Jul 06 19:28:45.045192 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.045155 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerStarted","Data":"0e01e04b34b778bf44ad9ccf72396075751a3856ee32dbd4c37118456e2e6057"} Jul 06 19:28:45.048167 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.048142 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" event={"ID":"66d06755-9fb1-4361-ae12-04286b7be77c","Type":"ContainerStarted","Data":"8cfed97fd16b8978325a78bb7cf4171be281f34339c4abf72c148c9cbf04586c"} Jul 06 19:28:45.048281 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.048201 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" event={"ID":"66d06755-9fb1-4361-ae12-04286b7be77c","Type":"ContainerStarted","Data":"47ba42639262aca94ccda83794e31b4a034a730bd26f657d67af3b230d67c103"} Jul 06 19:28:45.048281 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.048231 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" event={"ID":"66d06755-9fb1-4361-ae12-04286b7be77c","Type":"ContainerStarted","Data":"e90d77e048675ff6a387a50c20f637223a95c4b1a6bd10565b74c8da1d574e59"} Jul 06 19:28:45.050962 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.050931 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" event={"ID":"3c302db2-20ca-4d3c-91ff-5ce9c6565adf","Type":"ContainerStarted","Data":"ac0b3e5e2fa96796237c59ff66335d6b14a3991270acef2da3f3653d78dd6248"} Jul 06 19:28:45.087262 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.084966 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8gkq5" podStartSLOduration=1.964534244 podStartE2EDuration="4.084943217s" totalImagesPullingTime="2.120408973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:45.084022373 +0000 UTC m=+59.951697933" watchObservedRunningTime="2026-07-06 19:28:45.084943217 +0000 UTC m=+59.952618775" Jul 06 19:28:45.105166 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.105112 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-q8v7b" podStartSLOduration=1.860937785 podStartE2EDuration="4.10509427s" totalImagesPullingTime="2.244156485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:45.104044759 +0000 UTC m=+59.971720340" watchObservedRunningTime="2026-07-06 19:28:45.10509427 +0000 UTC m=+59.972769828" Jul 06 19:28:45.125695 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.125668 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5856747687-dl7fq"] Jul 06 19:28:45.205329 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.205275 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d5d96bddf-84kzv"] Jul 06 19:28:45.207695 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.207674 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.217183 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.217157 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 19:28:45.226254 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.226226 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d5d96bddf-84kzv"] Jul 06 19:28:45.289414 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.289377 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-oauth-serving-cert\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.289567 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.289505 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-serving-cert\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.289567 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.289533 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-oauth-config\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.289708 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.289620 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-config\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.289708 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.289664 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-trusted-ca-bundle\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.289787 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.289748 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74pbf\" (UniqueName: \"kubernetes.io/projected/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-kube-api-access-74pbf\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.289787 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.289773 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-service-ca\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.352316 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:45.352262 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f267dd3_a02f_46f5_8041_b2c06becc979.slice/crio-6d659b7345d628fdfe7640be42e3f1a86eadfe33d850c9aac378e1635bba2b0e WatchSource:0}: Error finding container 6d659b7345d628fdfe7640be42e3f1a86eadfe33d850c9aac378e1635bba2b0e: Status 404 returned error can't find the container with id 6d659b7345d628fdfe7640be42e3f1a86eadfe33d850c9aac378e1635bba2b0e Jul 06 19:28:45.390375 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.390345 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74pbf\" (UniqueName: \"kubernetes.io/projected/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-kube-api-access-74pbf\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.390489 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.390380 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-service-ca\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.390489 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.390409 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-oauth-serving-cert\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.390489 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.390465 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-serving-cert\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.390648 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.390492 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-oauth-config\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.390648 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.390543 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-config\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.390648 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.390583 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-trusted-ca-bundle\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.391126 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.391100 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-service-ca\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.391503 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.391482 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-config\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.391567 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.391540 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-oauth-serving-cert\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.391620 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.391601 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-trusted-ca-bundle\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.394022 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.393895 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-oauth-config\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.394135 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.394116 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-serving-cert\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.403752 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.403732 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74pbf\" (UniqueName: \"kubernetes.io/projected/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-kube-api-access-74pbf\") pod \"console-5d5d96bddf-84kzv\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.519735 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.519704 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:45.668907 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:45.668850 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d5d96bddf-84kzv"] Jul 06 19:28:45.669477 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:45.669450 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2fbcb00_8249_4c0a_a7f9_da60d04e7ad8.slice/crio-c92be9b46a10964c3c232a24f7d272e120390e162fce84e02b93f5bbb44e6bbe WatchSource:0}: Error finding container c92be9b46a10964c3c232a24f7d272e120390e162fce84e02b93f5bbb44e6bbe: Status 404 returned error can't find the container with id c92be9b46a10964c3c232a24f7d272e120390e162fce84e02b93f5bbb44e6bbe Jul 06 19:28:46.043829 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.043741 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5c5446d845-trq9w"] Jul 06 19:28:46.047572 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.047539 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.050843 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.050778 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 19:28:46.051009 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.050947 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 19:28:46.051009 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.050979 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-eqk57bg02l9sk\"" Jul 06 19:28:46.051009 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.051003 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bpvkf\"" Jul 06 19:28:46.051209 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.051089 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 19:28:46.051263 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.051221 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 19:28:46.057208 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.057181 2561 generic.go:356] "Generic (PLEG): container finished" podID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerID="db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd" exitCode=0 Jul 06 19:28:46.057344 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.057268 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerDied","Data":"db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd"} Jul 06 19:28:46.060026 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.059917 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5d96bddf-84kzv" event={"ID":"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8","Type":"ContainerStarted","Data":"3f644b2917578960b30ca7384e4ad164e4b4b2f6bbe006c5a262c6e47d013043"} Jul 06 19:28:46.060026 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.059962 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5d96bddf-84kzv" event={"ID":"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8","Type":"ContainerStarted","Data":"c92be9b46a10964c3c232a24f7d272e120390e162fce84e02b93f5bbb44e6bbe"} Jul 06 19:28:46.061685 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.061581 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" event={"ID":"4f267dd3-a02f-46f5-8041-b2c06becc979","Type":"ContainerStarted","Data":"6d659b7345d628fdfe7640be42e3f1a86eadfe33d850c9aac378e1635bba2b0e"} Jul 06 19:28:46.062178 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.062146 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c5446d845-trq9w"] Jul 06 19:28:46.064673 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.064618 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-62b95" event={"ID":"6b97f973-3964-4944-af29-163662153573","Type":"ContainerStarted","Data":"22f0c8ce51bc3952c0d46f862c9c719f0b82ecc27ddcbcfe6f18810587290461"} Jul 06 19:28:46.064774 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.064682 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-62b95" event={"ID":"6b97f973-3964-4944-af29-163662153573","Type":"ContainerStarted","Data":"d8b574dd271f204d8a5464d91be5026e26d0d5f537f02ce14c297dc9d5074746"} Jul 06 19:28:46.095765 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.095722 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e2009547-e35a-4b33-aed3-a89b82d24541-secret-metrics-server-client-certs\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.095993 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.095959 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e2009547-e35a-4b33-aed3-a89b82d24541-secret-metrics-server-tls\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.096117 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.096053 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e2009547-e35a-4b33-aed3-a89b82d24541-metrics-server-audit-profiles\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.096117 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.096110 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2009547-e35a-4b33-aed3-a89b82d24541-client-ca-bundle\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.096235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.096189 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2009547-e35a-4b33-aed3-a89b82d24541-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.096285 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.096274 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e2009547-e35a-4b33-aed3-a89b82d24541-audit-log\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.096406 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.096388 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k69xl\" (UniqueName: \"kubernetes.io/projected/e2009547-e35a-4b33-aed3-a89b82d24541-kube-api-access-k69xl\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.103101 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.103044 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-62b95" podStartSLOduration=3.942978137 podStartE2EDuration="5.103030661s" totalImagesPullingTime="1.062662686s" totalInitContainerRuntime="97.389838ms" isStatefulPod=true podCreationTimestamp="2026-07-06 19:28:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:46.102418451 +0000 UTC m=+60.970094007" watchObservedRunningTime="2026-07-06 19:28:46.103030661 +0000 UTC m=+60.970706217" Jul 06 19:28:46.152137 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.152072 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d5d96bddf-84kzv" podStartSLOduration=1.152048967 podStartE2EDuration="1.152048967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:46.151403805 +0000 UTC m=+61.019079362" watchObservedRunningTime="2026-07-06 19:28:46.152048967 +0000 UTC m=+61.019724523" Jul 06 19:28:46.197908 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.197875 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e2009547-e35a-4b33-aed3-a89b82d24541-audit-log\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.198398 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.198306 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k69xl\" (UniqueName: \"kubernetes.io/projected/e2009547-e35a-4b33-aed3-a89b82d24541-kube-api-access-k69xl\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.198510 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.198281 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e2009547-e35a-4b33-aed3-a89b82d24541-audit-log\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.198611 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.198586 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e2009547-e35a-4b33-aed3-a89b82d24541-secret-metrics-server-client-certs\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.198725 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.198705 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e2009547-e35a-4b33-aed3-a89b82d24541-secret-metrics-server-tls\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.198794 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.198764 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e2009547-e35a-4b33-aed3-a89b82d24541-metrics-server-audit-profiles\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.199520 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.198841 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2009547-e35a-4b33-aed3-a89b82d24541-client-ca-bundle\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.199520 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.198943 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2009547-e35a-4b33-aed3-a89b82d24541-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.199820 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.199799 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2009547-e35a-4b33-aed3-a89b82d24541-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.199922 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.199865 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e2009547-e35a-4b33-aed3-a89b82d24541-metrics-server-audit-profiles\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.201584 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.201325 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:46.201584 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.201375 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:46.201746 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.201725 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e2009547-e35a-4b33-aed3-a89b82d24541-secret-metrics-server-client-certs\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.201969 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.201942 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2009547-e35a-4b33-aed3-a89b82d24541-client-ca-bundle\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.202487 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.202464 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e2009547-e35a-4b33-aed3-a89b82d24541-secret-metrics-server-tls\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.207278 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.207255 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:46.207480 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.207462 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k69xl\" (UniqueName: \"kubernetes.io/projected/e2009547-e35a-4b33-aed3-a89b82d24541-kube-api-access-k69xl\") pod \"metrics-server-5c5446d845-trq9w\" (UID: \"e2009547-e35a-4b33-aed3-a89b82d24541\") " pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.301544 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.301466 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629"] Jul 06 19:28:46.320429 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.320401 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629"] Jul 06 19:28:46.320594 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.320527 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" Jul 06 19:28:46.323926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.323764 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-w5wnx\"" Jul 06 19:28:46.323926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.323919 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 19:28:46.360192 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.360161 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:28:46.401027 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.400996 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-4c629\" (UID: \"d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" Jul 06 19:28:46.401196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.401053 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgh8k\" (UniqueName: \"kubernetes.io/projected/d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d-kube-api-access-xgh8k\") pod \"monitoring-plugin-d5dcff7cf-4c629\" (UID: \"d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" Jul 06 19:28:46.502214 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.502177 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-4c629\" (UID: \"d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" Jul 06 19:28:46.502477 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.502235 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgh8k\" (UniqueName: \"kubernetes.io/projected/d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d-kube-api-access-xgh8k\") pod \"monitoring-plugin-d5dcff7cf-4c629\" (UID: \"d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" Jul 06 19:28:46.502477 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:46.502381 2561 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 06 19:28:46.502477 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:46.502454 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d-monitoring-plugin-cert podName:d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d nodeName:}" failed. No retries permitted until 2026-07-06 19:28:47.00243223 +0000 UTC m=+61.870107767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-4c629" (UID: "d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d") : secret "monitoring-plugin-cert" not found Jul 06 19:28:46.514436 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.514408 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgh8k\" (UniqueName: \"kubernetes.io/projected/d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d-kube-api-access-xgh8k\") pod \"monitoring-plugin-d5dcff7cf-4c629\" (UID: \"d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" Jul 06 19:28:46.517051 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.516977 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c5446d845-trq9w"] Jul 06 19:28:46.520968 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:46.520937 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2009547_e35a_4b33_aed3_a89b82d24541.slice/crio-97ede1ec666df5ce1490498378242481fe39d61cda654fc9fd009833a734e22b WatchSource:0}: Error finding container 97ede1ec666df5ce1490498378242481fe39d61cda654fc9fd009833a734e22b: Status 404 returned error can't find the container with id 97ede1ec666df5ce1490498378242481fe39d61cda654fc9fd009833a734e22b Jul 06 19:28:46.915222 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:46.915189 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mnlq8" Jul 06 19:28:47.006658 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.006617 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-4c629\" (UID: \"d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" Jul 06 19:28:47.006833 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:47.006784 2561 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 06 19:28:47.006897 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:28:47.006849 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d-monitoring-plugin-cert podName:d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d nodeName:}" failed. No retries permitted until 2026-07-06 19:28:48.006832522 +0000 UTC m=+62.874508079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-4c629" (UID: "d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d") : secret "monitoring-plugin-cert" not found Jul 06 19:28:47.071485 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.071452 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" event={"ID":"e2009547-e35a-4b33-aed3-a89b82d24541","Type":"ContainerStarted","Data":"97ede1ec666df5ce1490498378242481fe39d61cda654fc9fd009833a734e22b"} Jul 06 19:28:47.076515 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.076488 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:28:47.944940 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.944393 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:28:47.961765 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.960342 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:47.964486 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.964460 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 19:28:47.966853 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.964899 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 19:28:47.967019 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.964986 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 19:28:47.967214 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.965023 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 19:28:47.967318 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.965067 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 19:28:47.967379 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.965121 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7pjbf692hlv4\"" Jul 06 19:28:47.967432 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.965276 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 19:28:47.967496 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.966042 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 19:28:47.967561 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.966088 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 19:28:47.967718 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.966129 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 19:28:47.967718 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.966161 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fkkn5\"" Jul 06 19:28:47.967823 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.966202 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 19:28:47.967823 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.966244 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 19:28:47.967823 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.966259 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 19:28:47.968966 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.968920 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 19:28:47.973279 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.973258 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:28:47.973916 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:47.973895 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 19:28:48.020199 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020167 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020375 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020217 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c72d\" (UniqueName: \"kubernetes.io/projected/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-kube-api-access-2c72d\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020375 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020264 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020375 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020328 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020375 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020355 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020583 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020380 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020583 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020433 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-web-config\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020583 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020520 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020583 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020548 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020583 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020575 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020805 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020599 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-config\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020805 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020620 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-config-out\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020805 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020635 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020805 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020660 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020805 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020723 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.020805 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020772 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.021072 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020816 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-4c629\" (UID: \"d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" Jul 06 19:28:48.021072 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020845 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.021072 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020871 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.021072 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020897 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.021072 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.020931 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.026690 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.026660 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-4c629\" (UID: \"d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" Jul 06 19:28:48.084179 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.084106 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerStarted","Data":"f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae"} Jul 06 19:28:48.084634 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.084196 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerStarted","Data":"7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022"} Jul 06 19:28:48.084634 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.084242 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerStarted","Data":"ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1"} Jul 06 19:28:48.084634 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.084278 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerStarted","Data":"3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3"} Jul 06 19:28:48.087953 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.087880 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" event={"ID":"4f267dd3-a02f-46f5-8041-b2c06becc979","Type":"ContainerStarted","Data":"44f4c658e694aea0e0c8a85714fc71c5d777047a52214ac2417d5df6b054a860"} Jul 06 19:28:48.088055 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.087970 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" event={"ID":"4f267dd3-a02f-46f5-8041-b2c06becc979","Type":"ContainerStarted","Data":"9adfa2987cb71af5cca7e531554c8ad209608f6879bfafe85b6885e2987b9721"} Jul 06 19:28:48.088055 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.088016 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" event={"ID":"4f267dd3-a02f-46f5-8041-b2c06becc979","Type":"ContainerStarted","Data":"057b4ec20ff2ac607fa2b0de9dc044c97bef352b2db9de8a7a4c872fc8b761a8"} Jul 06 19:28:48.122406 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.122318 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.122406 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.122377 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c72d\" (UniqueName: \"kubernetes.io/projected/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-kube-api-access-2c72d\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.122586 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.122421 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.122586 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.122508 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.122586 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.122537 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.122586 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.122563 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.122776 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.122591 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-web-config\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.122863 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.122843 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123115 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123014 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123115 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123055 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123115 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123082 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123115 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123111 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-config\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123382 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123142 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-config-out\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123382 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123166 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123382 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123195 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123382 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123231 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123382 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123279 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123382 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123352 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123661 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123382 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123661 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123407 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123661 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123438 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123661 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123486 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.123661 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.123490 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.124145 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.124117 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.124255 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.124233 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.124723 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.124517 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.125623 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.125570 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.125724 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.125627 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.126652 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.126632 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-web-config\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.127158 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.126908 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.127339 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.127036 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.127339 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.127103 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.128309 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.128251 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.128309 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.128273 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-config\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.128474 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.128360 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.128874 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.128832 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.129376 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.129146 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-config-out\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.129376 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.129276 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.129884 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.129864 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.131414 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.131394 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c72d\" (UniqueName: \"kubernetes.io/projected/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-kube-api-access-2c72d\") pod \"prometheus-k8s-0\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.131525 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.131511 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" Jul 06 19:28:48.282752 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.282713 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:48.299261 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.299231 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629"] Jul 06 19:28:48.611229 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:48.611197 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b7c468_aa3f_4188_a9bc_5d09b43dcf2d.slice/crio-fef5cf03ec4fd6eb7c1bc1c836d1006f4e29fd0a5fb40fdc2b8e9992e3204f44 WatchSource:0}: Error finding container fef5cf03ec4fd6eb7c1bc1c836d1006f4e29fd0a5fb40fdc2b8e9992e3204f44: Status 404 returned error can't find the container with id fef5cf03ec4fd6eb7c1bc1c836d1006f4e29fd0a5fb40fdc2b8e9992e3204f44 Jul 06 19:28:48.797378 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:48.797245 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:28:48.804191 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:48.802382 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d77e7a2_b170_4ef7_a1a5_b05037522b1f.slice/crio-c65611a28b7de8874ad5bcfbbab9e722a200ef4492973730569bdcee5c23cc6a WatchSource:0}: Error finding container c65611a28b7de8874ad5bcfbbab9e722a200ef4492973730569bdcee5c23cc6a: Status 404 returned error can't find the container with id c65611a28b7de8874ad5bcfbbab9e722a200ef4492973730569bdcee5c23cc6a Jul 06 19:28:49.002087 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:49.002051 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-q9dm7" Jul 06 19:28:49.093041 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:49.093008 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" event={"ID":"d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d","Type":"ContainerStarted","Data":"fef5cf03ec4fd6eb7c1bc1c836d1006f4e29fd0a5fb40fdc2b8e9992e3204f44"} Jul 06 19:28:49.104581 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:49.104547 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerStarted","Data":"7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb"} Jul 06 19:28:49.106684 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:49.106652 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" event={"ID":"e2009547-e35a-4b33-aed3-a89b82d24541","Type":"ContainerStarted","Data":"b3068c4739b1e6c486464155d198dfa9172507a40eebb1cb1f5a5f1969c39213"} Jul 06 19:28:49.108753 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:49.108721 2561 generic.go:356] "Generic (PLEG): container finished" podID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerID="0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328" exitCode=0 Jul 06 19:28:49.108842 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:49.108767 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerDied","Data":"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328"} Jul 06 19:28:49.108842 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:49.108831 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerStarted","Data":"c65611a28b7de8874ad5bcfbbab9e722a200ef4492973730569bdcee5c23cc6a"} Jul 06 19:28:49.135853 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:49.135805 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" podStartSLOduration=0.998082923 podStartE2EDuration="3.135788389s" totalImagesPullingTime="2.137705466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:49.134326052 +0000 UTC m=+64.002001609" watchObservedRunningTime="2026-07-06 19:28:49.135788389 +0000 UTC m=+64.003463944" Jul 06 19:28:50.117583 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:50.117547 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerStarted","Data":"668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960"} Jul 06 19:28:50.120907 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:50.120870 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" event={"ID":"4f267dd3-a02f-46f5-8041-b2c06becc979","Type":"ContainerStarted","Data":"71b11293ec20ddbbbc825d5837d82000e2678dd2bbb1c57269b10c4b0214bccd"} Jul 06 19:28:50.121061 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:50.120936 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" event={"ID":"4f267dd3-a02f-46f5-8041-b2c06becc979","Type":"ContainerStarted","Data":"13e5d4574d948bf7eda31193298c094f3e8e3e55308371b7cf790bf20ff5f5c5"} Jul 06 19:28:50.121061 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:50.120987 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:50.121061 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:50.121023 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" event={"ID":"4f267dd3-a02f-46f5-8041-b2c06becc979","Type":"ContainerStarted","Data":"7c1fead081a47d3dc33340e984cb0244db69ef3fa19e1c2c3da0619ed9245354"} Jul 06 19:28:50.122580 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:50.122553 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" event={"ID":"d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d","Type":"ContainerStarted","Data":"dfe286fa3c8156d6d5a2de4857800f5cc638d0b7f1b8edf2000ed206afb1adff"} Jul 06 19:28:50.122894 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:50.122875 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" Jul 06 19:28:50.128283 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:50.128262 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" Jul 06 19:28:50.152486 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:50.152426 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.756913145 podStartE2EDuration="8.152401047s" totalImagesPullingTime="3.372277935s" totalInitContainerRuntime="23.209967ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:50.149419489 +0000 UTC m=+65.017095046" watchObservedRunningTime="2026-07-06 19:28:50.152401047 +0000 UTC m=+65.020076605" Jul 06 19:28:50.175605 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:50.175546 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" podStartSLOduration=2.752211005 podStartE2EDuration="6.175528199s" totalImagesPullingTime="3.423317194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:50.174415723 +0000 UTC m=+65.042091280" watchObservedRunningTime="2026-07-06 19:28:50.175528199 +0000 UTC m=+65.043203754" Jul 06 19:28:50.195023 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:50.194913 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-4c629" podStartSLOduration=2.866229697 podStartE2EDuration="4.194892368s" totalImagesPullingTime="1.328662671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:50.193603136 +0000 UTC m=+65.061278693" watchObservedRunningTime="2026-07-06 19:28:50.194892368 +0000 UTC m=+65.062567925" Jul 06 19:28:51.358258 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.358215 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs\") pod \"network-metrics-daemon-26crl\" (UID: \"477cd95b-0bfb-4470-8226-9ee5ed229f99\") " pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:51.361503 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.361477 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:28:51.371808 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.371775 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477cd95b-0bfb-4470-8226-9ee5ed229f99-metrics-certs\") pod \"network-metrics-daemon-26crl\" (UID: \"477cd95b-0bfb-4470-8226-9ee5ed229f99\") " pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:51.459075 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.459031 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:51.462818 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.462784 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:28:51.472052 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.472028 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d150905-fd23-4a84-9a45-729d76fd6652-original-pull-secret\") pod \"global-pull-secret-syncer-ddpss\" (UID: \"4d150905-fd23-4a84-9a45-729d76fd6652\") " pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:51.501330 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.501282 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddpss" Jul 06 19:28:51.510687 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.510663 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mjmjf\"" Jul 06 19:28:51.518454 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.518430 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-26crl" Jul 06 19:28:51.559728 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.559689 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkbtt\" (UniqueName: \"kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt\") pod \"network-check-target-28tgs\" (UID: \"46dc289d-2577-400d-84d5-8411d4225d46\") " pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:51.563608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.563582 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:28:51.575536 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.575504 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:28:51.584087 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.584057 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkbtt\" (UniqueName: \"kubernetes.io/projected/46dc289d-2577-400d-84d5-8411d4225d46-kube-api-access-vkbtt\") pod \"network-check-target-28tgs\" (UID: \"46dc289d-2577-400d-84d5-8411d4225d46\") " pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:51.814454 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.814422 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vdwjn\"" Jul 06 19:28:51.822416 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:51.822384 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:52.169258 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:52.169231 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-26crl"] Jul 06 19:28:52.181667 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:52.181634 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod477cd95b_0bfb_4470_8226_9ee5ed229f99.slice/crio-fa5a872fe2e8d820a9b2be645f2dbd586dbb9e18efab4b7884fb0b5f677e63ca WatchSource:0}: Error finding container fa5a872fe2e8d820a9b2be645f2dbd586dbb9e18efab4b7884fb0b5f677e63ca: Status 404 returned error can't find the container with id fa5a872fe2e8d820a9b2be645f2dbd586dbb9e18efab4b7884fb0b5f677e63ca Jul 06 19:28:52.418177 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:52.418101 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-28tgs"] Jul 06 19:28:52.419729 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:52.419693 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d150905_fd23_4a84_9a45_729d76fd6652.slice/crio-ec3c13839c048be4409561fd16e810f6aa10f35e2aad330034f72b7d602e21a0 WatchSource:0}: Error finding container ec3c13839c048be4409561fd16e810f6aa10f35e2aad330034f72b7d602e21a0: Status 404 returned error can't find the container with id ec3c13839c048be4409561fd16e810f6aa10f35e2aad330034f72b7d602e21a0 Jul 06 19:28:52.420208 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:52.420108 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ddpss"] Jul 06 19:28:52.420491 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:52.420465 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46dc289d_2577_400d_84d5_8411d4225d46.slice/crio-581ceb2d4f64d9f9689234495b0fb347a24f0ce3b398c0eb78d9ab5bca7608ca WatchSource:0}: Error finding container 581ceb2d4f64d9f9689234495b0fb347a24f0ce3b398c0eb78d9ab5bca7608ca: Status 404 returned error can't find the container with id 581ceb2d4f64d9f9689234495b0fb347a24f0ce3b398c0eb78d9ab5bca7608ca Jul 06 19:28:53.140454 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.140402 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ddpss" event={"ID":"4d150905-fd23-4a84-9a45-729d76fd6652","Type":"ContainerStarted","Data":"ec3c13839c048be4409561fd16e810f6aa10f35e2aad330034f72b7d602e21a0"} Jul 06 19:28:53.143710 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.143620 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-28tgs" event={"ID":"46dc289d-2577-400d-84d5-8411d4225d46","Type":"ContainerStarted","Data":"581ceb2d4f64d9f9689234495b0fb347a24f0ce3b398c0eb78d9ab5bca7608ca"} Jul 06 19:28:53.155645 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.155607 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerStarted","Data":"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e"} Jul 06 19:28:53.155809 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.155724 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerStarted","Data":"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c"} Jul 06 19:28:53.155809 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.155765 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerStarted","Data":"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302"} Jul 06 19:28:53.155809 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.155802 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerStarted","Data":"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff"} Jul 06 19:28:53.155971 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.155841 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerStarted","Data":"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9"} Jul 06 19:28:53.155971 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.155922 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerStarted","Data":"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb"} Jul 06 19:28:53.158671 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.158640 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-26crl" event={"ID":"477cd95b-0bfb-4470-8226-9ee5ed229f99","Type":"ContainerStarted","Data":"fa5a872fe2e8d820a9b2be645f2dbd586dbb9e18efab4b7884fb0b5f677e63ca"} Jul 06 19:28:53.184033 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.183966 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.191082739 podStartE2EDuration="6.183943529s" totalImagesPullingTime="2.970560825s" totalInitContainerRuntime="22.299965ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:53.183478723 +0000 UTC m=+68.051154312" watchObservedRunningTime="2026-07-06 19:28:53.183943529 +0000 UTC m=+68.051619083" Jul 06 19:28:53.283836 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.283800 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:28:53.427552 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.427281 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d5d96bddf-84kzv"] Jul 06 19:28:53.457121 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.457088 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-744b7f446c-srf8s"] Jul 06 19:28:53.490134 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.490081 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744b7f446c-srf8s"] Jul 06 19:28:53.490313 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.490213 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.584549 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.584511 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-service-ca\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.584707 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.584582 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtg84\" (UniqueName: \"kubernetes.io/projected/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-kube-api-access-mtg84\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.584707 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.584624 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-trusted-ca-bundle\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.584707 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.584666 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-config\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.584707 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.584702 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-serving-cert\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.584919 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.584768 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-oauth-config\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.584919 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.584810 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-oauth-serving-cert\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.685490 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.685364 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-config\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.685490 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.685414 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-serving-cert\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.685490 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.685473 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-oauth-config\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.685716 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.685509 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-oauth-serving-cert\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.685716 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.685539 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-service-ca\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.685716 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.685574 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtg84\" (UniqueName: \"kubernetes.io/projected/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-kube-api-access-mtg84\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.685716 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.685611 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-trusted-ca-bundle\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.686801 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.686770 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-service-ca\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.686896 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.686883 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-trusted-ca-bundle\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.687338 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.687310 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-oauth-serving-cert\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.687897 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.687870 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-config\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.689576 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.689535 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-serving-cert\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.691103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.691080 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-oauth-config\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.696278 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.696259 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtg84\" (UniqueName: \"kubernetes.io/projected/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-kube-api-access-mtg84\") pod \"console-744b7f446c-srf8s\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.803111 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.803073 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:28:53.988716 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:53.988591 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-744b7f446c-srf8s"] Jul 06 19:28:53.994241 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:28:53.994212 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eb7f2fd_8918_419b_b2a5_e2386e4a948e.slice/crio-76415643e94eeeada8c62026530bdca6adba06d29638fa3c0d0caeeb44734f0c WatchSource:0}: Error finding container 76415643e94eeeada8c62026530bdca6adba06d29638fa3c0d0caeeb44734f0c: Status 404 returned error can't find the container with id 76415643e94eeeada8c62026530bdca6adba06d29638fa3c0d0caeeb44734f0c Jul 06 19:28:54.168264 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:54.167952 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744b7f446c-srf8s" event={"ID":"0eb7f2fd-8918-419b-b2a5-e2386e4a948e","Type":"ContainerStarted","Data":"ecd1f79e008bd2f73ce1cca7357c4817f6979a2be649fff2a7cee84ee40b5d3a"} Jul 06 19:28:54.168264 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:54.168022 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744b7f446c-srf8s" event={"ID":"0eb7f2fd-8918-419b-b2a5-e2386e4a948e","Type":"ContainerStarted","Data":"76415643e94eeeada8c62026530bdca6adba06d29638fa3c0d0caeeb44734f0c"} Jul 06 19:28:54.170916 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:54.170403 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-26crl" event={"ID":"477cd95b-0bfb-4470-8226-9ee5ed229f99","Type":"ContainerStarted","Data":"a799b34fc66f24f296c59f004237bb0ad91aecbd2c31d7ffaf450dea966fe841"} Jul 06 19:28:54.170916 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:54.170476 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-26crl" event={"ID":"477cd95b-0bfb-4470-8226-9ee5ed229f99","Type":"ContainerStarted","Data":"ba51477b7c8b873c6a4582261d56bab8bf3fd9171e625489907ef1ce304f1269"} Jul 06 19:28:54.189432 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:54.189366 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-744b7f446c-srf8s" podStartSLOduration=1.189343725 podStartE2EDuration="1.189343725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:54.188457585 +0000 UTC m=+69.056133142" watchObservedRunningTime="2026-07-06 19:28:54.189343725 +0000 UTC m=+69.057019282" Jul 06 19:28:54.208547 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:54.208475 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-26crl" podStartSLOduration=67.026063746 podStartE2EDuration="1m8.208453667s" totalImagesPullingTime="1.182389921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:54.207332179 +0000 UTC m=+69.075007736" watchObservedRunningTime="2026-07-06 19:28:54.208453667 +0000 UTC m=+69.076129227" Jul 06 19:28:55.520311 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:55.520221 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:28:56.133694 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:56.133670 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5856747687-dl7fq" Jul 06 19:28:58.195974 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:58.195940 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-28tgs" event={"ID":"46dc289d-2577-400d-84d5-8411d4225d46","Type":"ContainerStarted","Data":"11b43a642d9b17f9af16aabb6b6cdd297ef89afb56d2daf42a25501da28f5fd4"} Jul 06 19:28:58.196496 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:58.196043 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:28:58.197439 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:58.197417 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ddpss" event={"ID":"4d150905-fd23-4a84-9a45-729d76fd6652","Type":"ContainerStarted","Data":"6c86d2b13a70a30c3889ddd9ac2a0b6ae430a067bd49e06704bfd965f8e644dc"} Jul 06 19:28:58.217427 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:58.217379 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-28tgs" podStartSLOduration=66.264532265 podStartE2EDuration="1m11.217363603s" totalImagesPullingTime="4.952831338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:58.215714302 +0000 UTC m=+73.083389859" watchObservedRunningTime="2026-07-06 19:28:58.217363603 +0000 UTC m=+73.085039425" Jul 06 19:28:58.233887 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:28:58.233835 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ddpss" podStartSLOduration=66.273219819 podStartE2EDuration="1m11.233819962s" totalImagesPullingTime="4.960600143s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:58.232746846 +0000 UTC m=+73.100422399" watchObservedRunningTime="2026-07-06 19:28:58.233819962 +0000 UTC m=+73.101495518" Jul 06 19:29:03.804044 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:03.804007 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:29:03.804559 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:03.804076 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:29:03.808777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:03.808752 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:29:04.228683 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:04.228657 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:29:04.278402 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:04.278373 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c646d46d5-bfp57"] Jul 06 19:29:06.360599 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:06.360557 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:29:06.360972 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:06.360623 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:29:18.458731 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:18.458593 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5d5d96bddf-84kzv" podUID="c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8" containerName="console" containerID="cri-o://3f644b2917578960b30ca7384e4ad164e4b4b2f6bbe006c5a262c6e47d013043" gracePeriod=15 Jul 06 19:29:19.236324 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.236283 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:29:19.287749 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.287719 2561 generic.go:356] "Generic (PLEG): container finished" podID="c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8" containerID="3f644b2917578960b30ca7384e4ad164e4b4b2f6bbe006c5a262c6e47d013043" exitCode=0 Jul 06 19:29:19.287901 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.287765 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5d96bddf-84kzv" Jul 06 19:29:19.287901 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.287787 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5d96bddf-84kzv" event={"ID":"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8","Type":"ContainerDied","Data":"3f644b2917578960b30ca7384e4ad164e4b4b2f6bbe006c5a262c6e47d013043"} Jul 06 19:29:19.287901 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.287853 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5d96bddf-84kzv" event={"ID":"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8","Type":"ContainerDied","Data":"c92be9b46a10964c3c232a24f7d272e120390e162fce84e02b93f5bbb44e6bbe"} Jul 06 19:29:19.287901 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.287874 2561 scope.go:117] "RemoveContainer" containerID="3f644b2917578960b30ca7384e4ad164e4b4b2f6bbe006c5a262c6e47d013043" Jul 06 19:29:19.295979 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.295960 2561 scope.go:117] "RemoveContainer" containerID="3f644b2917578960b30ca7384e4ad164e4b4b2f6bbe006c5a262c6e47d013043" Jul 06 19:29:19.296279 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:29:19.296261 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f644b2917578960b30ca7384e4ad164e4b4b2f6bbe006c5a262c6e47d013043\": container with ID starting with 3f644b2917578960b30ca7384e4ad164e4b4b2f6bbe006c5a262c6e47d013043 not found: ID does not exist" containerID="3f644b2917578960b30ca7384e4ad164e4b4b2f6bbe006c5a262c6e47d013043" Jul 06 19:29:19.296352 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.296311 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f644b2917578960b30ca7384e4ad164e4b4b2f6bbe006c5a262c6e47d013043"} err="failed to get container status \"3f644b2917578960b30ca7384e4ad164e4b4b2f6bbe006c5a262c6e47d013043\": rpc error: code = NotFound desc = could not find container \"3f644b2917578960b30ca7384e4ad164e4b4b2f6bbe006c5a262c6e47d013043\": container with ID starting with 3f644b2917578960b30ca7384e4ad164e4b4b2f6bbe006c5a262c6e47d013043 not found: ID does not exist" Jul 06 19:29:19.319937 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.319913 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-oauth-serving-cert\") pod \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " Jul 06 19:29:19.320063 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.319952 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-config\") pod \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " Jul 06 19:29:19.320063 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.319998 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-serving-cert\") pod \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " Jul 06 19:29:19.320063 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.320040 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-service-ca\") pod \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " Jul 06 19:29:19.320182 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.320103 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-oauth-config\") pod \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " Jul 06 19:29:19.320182 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.320139 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74pbf\" (UniqueName: \"kubernetes.io/projected/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-kube-api-access-74pbf\") pod \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " Jul 06 19:29:19.320283 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.320206 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-trusted-ca-bundle\") pod \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\" (UID: \"c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8\") " Jul 06 19:29:19.320366 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.320347 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8" (UID: "c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:29:19.320587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.320538 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-config" (OuterVolumeSpecName: "console-config") pod "c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8" (UID: "c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:29:19.320587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.320571 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-service-ca" (OuterVolumeSpecName: "service-ca") pod "c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8" (UID: "c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:29:19.320749 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.320660 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8" (UID: "c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:29:19.320789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.320745 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-oauth-serving-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:29:19.320789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.320763 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-config\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:29:19.320789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.320775 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-service-ca\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:29:19.320789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.320783 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-trusted-ca-bundle\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:29:19.322637 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.322610 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-kube-api-access-74pbf" (OuterVolumeSpecName: "kube-api-access-74pbf") pod "c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8" (UID: "c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8"). InnerVolumeSpecName "kube-api-access-74pbf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:29:19.322762 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.322743 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8" (UID: "c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:29:19.322850 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.322827 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8" (UID: "c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:29:19.421472 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.421429 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-oauth-config\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:29:19.421472 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.421471 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74pbf\" (UniqueName: \"kubernetes.io/projected/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-kube-api-access-74pbf\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:29:19.421668 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.421486 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8-console-serving-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:29:19.614571 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.614541 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d5d96bddf-84kzv"] Jul 06 19:29:19.619183 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.619159 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d5d96bddf-84kzv"] Jul 06 19:29:19.686347 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:19.686258 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8" path="/var/lib/kubelet/pods/c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8/volumes" Jul 06 19:29:26.366227 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:26.366198 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:29:26.369978 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:26.369961 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5c5446d845-trq9w" Jul 06 19:29:29.204105 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.204073 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-28tgs" Jul 06 19:29:29.299757 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.299693 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7c646d46d5-bfp57" podUID="c819dedf-7c04-4003-8555-fdb5e5857f22" containerName="console" containerID="cri-o://cfa4992941b39b81802f6b016c23a862ad7732b431fd4685fb7d3708e1c2d6f0" gracePeriod=15 Jul 06 19:29:29.578539 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.578501 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:29:29.711052 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.711016 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c819dedf-7c04-4003-8555-fdb5e5857f22-console-oauth-config\") pod \"c819dedf-7c04-4003-8555-fdb5e5857f22\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " Jul 06 19:29:29.711244 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.711081 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-service-ca\") pod \"c819dedf-7c04-4003-8555-fdb5e5857f22\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " Jul 06 19:29:29.711244 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.711138 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-oauth-serving-cert\") pod \"c819dedf-7c04-4003-8555-fdb5e5857f22\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " Jul 06 19:29:29.711244 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.711178 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c819dedf-7c04-4003-8555-fdb5e5857f22-console-serving-cert\") pod \"c819dedf-7c04-4003-8555-fdb5e5857f22\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " Jul 06 19:29:29.711244 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.711210 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6v7l\" (UniqueName: \"kubernetes.io/projected/c819dedf-7c04-4003-8555-fdb5e5857f22-kube-api-access-b6v7l\") pod \"c819dedf-7c04-4003-8555-fdb5e5857f22\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " Jul 06 19:29:29.711481 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.711286 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-console-config\") pod \"c819dedf-7c04-4003-8555-fdb5e5857f22\" (UID: \"c819dedf-7c04-4003-8555-fdb5e5857f22\") " Jul 06 19:29:29.711548 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.711524 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c819dedf-7c04-4003-8555-fdb5e5857f22" (UID: "c819dedf-7c04-4003-8555-fdb5e5857f22"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:29:29.711621 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.711606 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-oauth-serving-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:29:29.711677 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.711559 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-service-ca" (OuterVolumeSpecName: "service-ca") pod "c819dedf-7c04-4003-8555-fdb5e5857f22" (UID: "c819dedf-7c04-4003-8555-fdb5e5857f22"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:29:29.711788 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.711721 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-console-config" (OuterVolumeSpecName: "console-config") pod "c819dedf-7c04-4003-8555-fdb5e5857f22" (UID: "c819dedf-7c04-4003-8555-fdb5e5857f22"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:29:29.713690 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.713668 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c819dedf-7c04-4003-8555-fdb5e5857f22-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c819dedf-7c04-4003-8555-fdb5e5857f22" (UID: "c819dedf-7c04-4003-8555-fdb5e5857f22"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:29:29.713690 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.713670 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c819dedf-7c04-4003-8555-fdb5e5857f22-kube-api-access-b6v7l" (OuterVolumeSpecName: "kube-api-access-b6v7l") pod "c819dedf-7c04-4003-8555-fdb5e5857f22" (UID: "c819dedf-7c04-4003-8555-fdb5e5857f22"). InnerVolumeSpecName "kube-api-access-b6v7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:29:29.713840 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.713738 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c819dedf-7c04-4003-8555-fdb5e5857f22-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c819dedf-7c04-4003-8555-fdb5e5857f22" (UID: "c819dedf-7c04-4003-8555-fdb5e5857f22"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:29:29.812690 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.812593 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c819dedf-7c04-4003-8555-fdb5e5857f22-console-serving-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:29:29.812690 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.812626 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6v7l\" (UniqueName: \"kubernetes.io/projected/c819dedf-7c04-4003-8555-fdb5e5857f22-kube-api-access-b6v7l\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:29:29.812690 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.812639 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-console-config\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:29:29.812690 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.812651 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c819dedf-7c04-4003-8555-fdb5e5857f22-console-oauth-config\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:29:29.812690 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:29.812664 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c819dedf-7c04-4003-8555-fdb5e5857f22-service-ca\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:29:30.340102 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:30.340070 2561 generic.go:356] "Generic (PLEG): container finished" podID="c819dedf-7c04-4003-8555-fdb5e5857f22" containerID="cfa4992941b39b81802f6b016c23a862ad7732b431fd4685fb7d3708e1c2d6f0" exitCode=0 Jul 06 19:29:30.340574 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:30.340112 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c646d46d5-bfp57" Jul 06 19:29:30.340574 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:30.340155 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c646d46d5-bfp57" event={"ID":"c819dedf-7c04-4003-8555-fdb5e5857f22","Type":"ContainerDied","Data":"cfa4992941b39b81802f6b016c23a862ad7732b431fd4685fb7d3708e1c2d6f0"} Jul 06 19:29:30.340574 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:30.340209 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c646d46d5-bfp57" event={"ID":"c819dedf-7c04-4003-8555-fdb5e5857f22","Type":"ContainerDied","Data":"1b73c21d3cb7b6cbed34f31b2f6403a6997c647b1f884a0fceee76e4f152f81c"} Jul 06 19:29:30.340574 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:30.340235 2561 scope.go:117] "RemoveContainer" containerID="cfa4992941b39b81802f6b016c23a862ad7732b431fd4685fb7d3708e1c2d6f0" Jul 06 19:29:30.349313 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:30.349280 2561 scope.go:117] "RemoveContainer" containerID="cfa4992941b39b81802f6b016c23a862ad7732b431fd4685fb7d3708e1c2d6f0" Jul 06 19:29:30.349570 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:29:30.349550 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfa4992941b39b81802f6b016c23a862ad7732b431fd4685fb7d3708e1c2d6f0\": container with ID starting with cfa4992941b39b81802f6b016c23a862ad7732b431fd4685fb7d3708e1c2d6f0 not found: ID does not exist" containerID="cfa4992941b39b81802f6b016c23a862ad7732b431fd4685fb7d3708e1c2d6f0" Jul 06 19:29:30.349636 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:30.349577 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfa4992941b39b81802f6b016c23a862ad7732b431fd4685fb7d3708e1c2d6f0"} err="failed to get container status \"cfa4992941b39b81802f6b016c23a862ad7732b431fd4685fb7d3708e1c2d6f0\": rpc error: code = NotFound desc = could not find container \"cfa4992941b39b81802f6b016c23a862ad7732b431fd4685fb7d3708e1c2d6f0\": container with ID starting with cfa4992941b39b81802f6b016c23a862ad7732b431fd4685fb7d3708e1c2d6f0 not found: ID does not exist" Jul 06 19:29:30.363943 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:30.363913 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c646d46d5-bfp57"] Jul 06 19:29:30.368758 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:30.368734 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c646d46d5-bfp57"] Jul 06 19:29:31.686190 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:31.686157 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c819dedf-7c04-4003-8555-fdb5e5857f22" path="/var/lib/kubelet/pods/c819dedf-7c04-4003-8555-fdb5e5857f22/volumes" Jul 06 19:29:48.283919 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:48.283882 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:48.302917 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:48.302894 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:48.430274 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:29:48.430248 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:29:58.483910 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:29:58.483873 2561 controller.go:195] "Failed to update lease" err="Put \"https://acd028595317847db81edfdd98eff41b-54b36feb6ca41253.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-141-33.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 06 19:30:08.484566 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:08.484524 2561 controller.go:195] "Failed to update lease" err="Put \"https://acd028595317847db81edfdd98eff41b-54b36feb6ca41253.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-141-33.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 06 19:30:18.485333 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:18.485276 2561 controller.go:195] "Failed to update lease" err="Put \"https://acd028595317847db81edfdd98eff41b-54b36feb6ca41253.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-141-33.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 06 19:30:18.732799 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:18.732753 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:30:08Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:30:08Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:30:08Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:30:08Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-141-33.ec2.internal\": Patch \"https://acd028595317847db81edfdd98eff41b-54b36feb6ca41253.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-141-33.ec2.internal/status?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 06 19:30:23.147382 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147344 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147382 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147371 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147416 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fkkn5\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147414 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147440 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147443 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147480 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bp44x\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147487 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nmc6g\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147499 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147502 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147534 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-dockercfg-cj9rh\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147544 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147557 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147555 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147580 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147611 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147631 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147645 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147660 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.147777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147689 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147698 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147704 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147711 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147727 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bbbsc\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147732 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147764 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147763 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147789 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qvtqt\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147796 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147818 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6kw96\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147822 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147819 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147841 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147842 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147856 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147870 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147878 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fxbq4\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.148550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147923 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7pjbf692hlv4\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147943 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147958 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-oauth-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147973 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147982 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.147987 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148014 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148015 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148026 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148045 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148048 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-w5wnx\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148062 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148066 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148065 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9hz74\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148091 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148103 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s6dss\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148107 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"original-pull-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148104 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148129 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148135 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148149 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148151 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148159 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148172 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148180 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148186 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148199 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mjmjf\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148195 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148217 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148238 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"default-dockercfg-h99cc\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148243 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148248 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z9rrt\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148263 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"default-dockercfg-gwvng\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148279 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148283 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148310 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.149638 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148315 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148325 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148328 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148329 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:23.148285 2561 controller.go:195] "Failed to update lease" err="Put \"https://acd028595317847db81edfdd98eff41b-54b36feb6ca41253.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-141-33.ec2.internal?timeout=10s\": http2: client connection lost" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148347 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148346 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148354 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148364 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"service-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148369 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148385 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148390 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148397 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148412 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148418 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148419 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-eqk57bg02l9sk\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148420 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7g8bw\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148425 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148443 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148448 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mrjcz\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148452 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148463 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148469 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148472 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148475 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148338 2561 reflector.go:571] "Warning: watch ended with error" reflector="pkg/kubelet/config/apiserver.go:66" type="*v1.Pod" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148482 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148489 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qg2nr\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148489 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148510 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148514 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148508 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sq7z5\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148527 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-nx7pq\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148510 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148537 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cck9p\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148540 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.150672 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148549 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148534 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148559 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148559 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148579 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148575 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bpvkf\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148589 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148594 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148586 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148605 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6h368rfb0l1j8\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148591 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148612 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148608 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148625 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6s68c\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148575 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vdwjn\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148614 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-agent\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.151275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.148581 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 19:30:23.979642 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.979614 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h99cc\"" Jul 06 19:30:23.997623 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.997605 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 19:30:23.997712 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:23.997605 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 19:30:24.011856 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.011835 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 19:30:24.013605 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.013591 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 19:30:24.016865 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.016848 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 19:30:24.016906 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.016848 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gwvng\"" Jul 06 19:30:24.023861 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.023847 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 19:30:24.031953 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.031938 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 19:30:24.034547 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.034531 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 19:30:24.038663 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.038646 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 19:30:24.040461 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.040447 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 19:30:24.052745 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.052729 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 19:30:24.052800 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.052757 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 19:30:24.060826 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.060807 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 19:30:24.062076 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.062063 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 19:30:24.066465 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.066452 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 19:30:24.070904 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.070890 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 19:30:24.080062 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.080048 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 19:30:24.080864 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.080852 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 19:30:24.110735 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.110712 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 19:30:24.110825 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.110715 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 19:30:24.125653 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.125633 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 19:30:24.125927 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.125913 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:30:24.130260 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.130247 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z9rrt\"" Jul 06 19:30:24.131211 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.131198 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 19:30:24.135684 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.135670 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 19:30:24.140262 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.140247 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cj9rh\"" Jul 06 19:30:24.144642 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.144630 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 19:30:24.153659 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.153644 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 19:30:24.159029 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.159016 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 19:30:24.162505 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.162487 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 19:30:24.162583 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.162487 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bpvkf\"" Jul 06 19:30:24.162773 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.162760 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fxbq4\"" Jul 06 19:30:24.173642 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.173624 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sq7z5\"" Jul 06 19:30:24.186916 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.186892 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 19:30:24.188977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.188958 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 19:30:24.206894 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.206874 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 19:30:24.206998 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.206907 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 19:30:24.221463 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.221439 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6s68c\"" Jul 06 19:30:24.231561 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.231495 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 19:30:24.233222 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.233200 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7g8bw\"" Jul 06 19:30:24.234721 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.234701 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 19:30:24.237409 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.237392 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 19:30:24.240050 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.240037 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 19:30:24.257883 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.257864 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 19:30:24.259255 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.259238 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 19:30:24.265724 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.265707 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 19:30:24.275533 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.275518 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 19:30:24.279917 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.279902 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 19:30:24.286656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.286641 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-eqk57bg02l9sk\"" Jul 06 19:30:24.311002 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.310983 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:30:24.321305 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.321267 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 19:30:24.332152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.332135 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fkkn5\"" Jul 06 19:30:24.337720 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.337705 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 19:30:24.350296 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.350277 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 19:30:24.357258 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.357244 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 19:30:24.366882 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.366867 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 19:30:24.370214 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.370200 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 19:30:24.375869 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.375854 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nmc6g\"" Jul 06 19:30:24.385076 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.385059 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 19:30:24.385125 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.385077 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 19:30:24.398775 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.398753 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 19:30:24.414347 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.414327 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 19:30:24.414559 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.414531 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 19:30:24.417981 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.417967 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-w5wnx\"" Jul 06 19:30:24.419613 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.419601 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 19:30:24.422884 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.422871 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9hz74\"" Jul 06 19:30:24.431324 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.431311 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 19:30:24.434048 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.434028 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 19:30:24.435520 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.435506 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 19:30:24.435598 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.435584 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 19:30:24.445255 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.445238 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 19:30:24.448883 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.448870 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 19:30:24.450695 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.450683 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 19:30:24.454637 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.454623 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:30:24.467263 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.467246 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 19:30:24.486639 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.486593 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 19:30:24.496404 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.496389 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 19:30:24.501688 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.501673 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 19:30:24.502301 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.502276 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 19:30:24.503497 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.503485 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 19:30:24.510780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.510767 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 19:30:24.517140 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.517125 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 19:30:24.528607 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.528588 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:30:24.544741 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.544723 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 19:30:24.545709 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.545695 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 19:30:24.547842 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.547828 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 19:30:24.550161 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.550147 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bp44x\"" Jul 06 19:30:24.563749 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.563731 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 19:30:24.583167 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.583147 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qvtqt\"" Jul 06 19:30:24.583249 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.583149 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6kw96\"" Jul 06 19:30:24.593320 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.593284 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 19:30:24.608893 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.608873 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s6dss\"" Jul 06 19:30:24.611820 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.611804 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 19:30:24.616506 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.616491 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 19:30:24.617869 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.617857 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 19:30:24.624401 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.624387 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bbbsc\"" Jul 06 19:30:24.650612 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.650587 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 19:30:24.650990 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.650977 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 19:30:24.658908 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.658890 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:30:24.659034 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.659019 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 19:30:24.659034 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.659030 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:30:24.668128 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.668107 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 19:30:24.674523 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.674507 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6h368rfb0l1j8\"" Jul 06 19:30:24.683530 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.683507 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 19:30:24.693764 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.693745 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mrjcz\"" Jul 06 19:30:24.698523 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.698508 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 19:30:24.701373 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.701359 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 19:30:24.702121 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.702108 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7pjbf692hlv4\"" Jul 06 19:30:24.719465 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.719434 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 19:30:24.719816 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.719800 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 19:30:24.721247 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.721228 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 19:30:24.727916 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.727899 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qg2nr\"" Jul 06 19:30:24.728065 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.728053 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cck9p\"" Jul 06 19:30:24.734829 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.734814 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 19:30:24.738331 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.738270 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-nx7pq\"" Jul 06 19:30:24.739427 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.739414 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 19:30:24.750870 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.750854 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 19:30:24.952450 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.952413 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:30:24.963465 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:24.963433 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mjmjf\"" Jul 06 19:30:25.263510 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:25.263480 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:30:25.470178 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:25.470146 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:30:25.535066 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:25.534989 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vdwjn\"" Jul 06 19:30:26.068446 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:26.068412 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:30:32.002519 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.002484 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:30:32.003113 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.002867 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="alertmanager" containerID="cri-o://3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3" gracePeriod=120 Jul 06 19:30:32.003113 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.002918 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="kube-rbac-proxy-metric" containerID="cri-o://7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb" gracePeriod=120 Jul 06 19:30:32.003113 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.002967 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="prom-label-proxy" containerID="cri-o://668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960" gracePeriod=120 Jul 06 19:30:32.003113 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.002998 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="config-reloader" containerID="cri-o://ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1" gracePeriod=120 Jul 06 19:30:32.003113 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.002993 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="kube-rbac-proxy" containerID="cri-o://f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae" gracePeriod=120 Jul 06 19:30:32.003113 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.002975 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="kube-rbac-proxy-web" containerID="cri-o://7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022" gracePeriod=120 Jul 06 19:30:32.586277 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.586235 2561 generic.go:356] "Generic (PLEG): container finished" podID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerID="668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960" exitCode=0 Jul 06 19:30:32.586277 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.586264 2561 generic.go:356] "Generic (PLEG): container finished" podID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerID="f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae" exitCode=0 Jul 06 19:30:32.586277 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.586270 2561 generic.go:356] "Generic (PLEG): container finished" podID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerID="ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1" exitCode=0 Jul 06 19:30:32.586277 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.586276 2561 generic.go:356] "Generic (PLEG): container finished" podID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerID="3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3" exitCode=0 Jul 06 19:30:32.586561 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.586327 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerDied","Data":"668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960"} Jul 06 19:30:32.586561 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.586425 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerDied","Data":"f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae"} Jul 06 19:30:32.586561 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.586455 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerDied","Data":"ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1"} Jul 06 19:30:32.586561 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:32.586479 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerDied","Data":"3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3"} Jul 06 19:30:33.255044 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.255015 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.362560 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.362468 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy-web\") pod \"a2095283-08a9-4fc7-b28f-46552ea28b04\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " Jul 06 19:30:33.362560 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.362512 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdmtr\" (UniqueName: \"kubernetes.io/projected/a2095283-08a9-4fc7-b28f-46552ea28b04-kube-api-access-wdmtr\") pod \"a2095283-08a9-4fc7-b28f-46552ea28b04\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " Jul 06 19:30:33.362560 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.362528 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-cluster-tls-config\") pod \"a2095283-08a9-4fc7-b28f-46552ea28b04\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " Jul 06 19:30:33.362560 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.362546 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-web-config\") pod \"a2095283-08a9-4fc7-b28f-46552ea28b04\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " Jul 06 19:30:33.362896 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.362576 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2095283-08a9-4fc7-b28f-46552ea28b04-tls-assets\") pod \"a2095283-08a9-4fc7-b28f-46552ea28b04\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " Jul 06 19:30:33.362896 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.362616 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-main-db\") pod \"a2095283-08a9-4fc7-b28f-46552ea28b04\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " Jul 06 19:30:33.362896 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.362633 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-config-volume\") pod \"a2095283-08a9-4fc7-b28f-46552ea28b04\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " Jul 06 19:30:33.362896 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.362655 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-trusted-ca-bundle\") pod \"a2095283-08a9-4fc7-b28f-46552ea28b04\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " Jul 06 19:30:33.362896 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.362704 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2095283-08a9-4fc7-b28f-46552ea28b04-config-out\") pod \"a2095283-08a9-4fc7-b28f-46552ea28b04\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " Jul 06 19:30:33.362896 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.362728 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy\") pod \"a2095283-08a9-4fc7-b28f-46552ea28b04\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " Jul 06 19:30:33.362896 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.362777 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a2095283-08a9-4fc7-b28f-46552ea28b04\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " Jul 06 19:30:33.362896 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.362803 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-metrics-client-ca\") pod \"a2095283-08a9-4fc7-b28f-46552ea28b04\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " Jul 06 19:30:33.362896 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.362837 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-main-tls\") pod \"a2095283-08a9-4fc7-b28f-46552ea28b04\" (UID: \"a2095283-08a9-4fc7-b28f-46552ea28b04\") " Jul 06 19:30:33.363324 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.362978 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a2095283-08a9-4fc7-b28f-46552ea28b04" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:30:33.363324 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.363156 2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-main-db\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:33.363737 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.363655 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a2095283-08a9-4fc7-b28f-46552ea28b04" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:33.363892 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.363864 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a2095283-08a9-4fc7-b28f-46552ea28b04" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:33.365850 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.365818 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a2095283-08a9-4fc7-b28f-46552ea28b04" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:33.366222 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.366197 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a2095283-08a9-4fc7-b28f-46552ea28b04" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:33.366391 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.366369 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2095283-08a9-4fc7-b28f-46552ea28b04-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a2095283-08a9-4fc7-b28f-46552ea28b04" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:30:33.366692 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.366651 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a2095283-08a9-4fc7-b28f-46552ea28b04" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:33.366769 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.366743 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2095283-08a9-4fc7-b28f-46552ea28b04-kube-api-access-wdmtr" (OuterVolumeSpecName: "kube-api-access-wdmtr") pod "a2095283-08a9-4fc7-b28f-46552ea28b04" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04"). InnerVolumeSpecName "kube-api-access-wdmtr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:30:33.366826 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.366775 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-config-volume" (OuterVolumeSpecName: "config-volume") pod "a2095283-08a9-4fc7-b28f-46552ea28b04" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:33.366881 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.366854 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a2095283-08a9-4fc7-b28f-46552ea28b04" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:33.367644 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.367626 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2095283-08a9-4fc7-b28f-46552ea28b04-config-out" (OuterVolumeSpecName: "config-out") pod "a2095283-08a9-4fc7-b28f-46552ea28b04" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:30:33.370546 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.370526 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a2095283-08a9-4fc7-b28f-46552ea28b04" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:33.376736 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.376711 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-web-config" (OuterVolumeSpecName: "web-config") pod "a2095283-08a9-4fc7-b28f-46552ea28b04" (UID: "a2095283-08a9-4fc7-b28f-46552ea28b04"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:33.464580 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.464539 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:33.464580 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.464575 2561 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-metrics-client-ca\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:33.464796 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.464590 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-main-tls\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:33.464796 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.464603 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:33.464796 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.464616 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdmtr\" (UniqueName: \"kubernetes.io/projected/a2095283-08a9-4fc7-b28f-46552ea28b04-kube-api-access-wdmtr\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:33.464796 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.464627 2561 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-cluster-tls-config\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:33.464796 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.464639 2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-web-config\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:33.464796 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.464649 2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a2095283-08a9-4fc7-b28f-46552ea28b04-tls-assets\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:33.464796 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.464661 2561 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-config-volume\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:33.464796 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.464671 2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2095283-08a9-4fc7-b28f-46552ea28b04-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:33.464796 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.464685 2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a2095283-08a9-4fc7-b28f-46552ea28b04-config-out\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:33.464796 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.464698 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a2095283-08a9-4fc7-b28f-46552ea28b04-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:33.593073 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.593034 2561 generic.go:356] "Generic (PLEG): container finished" podID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerID="7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb" exitCode=0 Jul 06 19:30:33.593073 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.593064 2561 generic.go:356] "Generic (PLEG): container finished" podID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerID="7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022" exitCode=0 Jul 06 19:30:33.593263 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.593115 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerDied","Data":"7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb"} Jul 06 19:30:33.593263 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.593185 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerDied","Data":"7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022"} Jul 06 19:30:33.593263 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.593125 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.593263 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.593215 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a2095283-08a9-4fc7-b28f-46552ea28b04","Type":"ContainerDied","Data":"0e01e04b34b778bf44ad9ccf72396075751a3856ee32dbd4c37118456e2e6057"} Jul 06 19:30:33.593263 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.593245 2561 scope.go:117] "RemoveContainer" containerID="668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960" Jul 06 19:30:33.601971 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.601954 2561 scope.go:117] "RemoveContainer" containerID="7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb" Jul 06 19:30:33.608612 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.608596 2561 scope.go:117] "RemoveContainer" containerID="f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae" Jul 06 19:30:33.614889 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.614873 2561 scope.go:117] "RemoveContainer" containerID="7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022" Jul 06 19:30:33.619209 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.619185 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:30:33.622959 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.622931 2561 scope.go:117] "RemoveContainer" containerID="ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1" Jul 06 19:30:33.623066 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.623016 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:30:33.630005 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.629979 2561 scope.go:117] "RemoveContainer" containerID="3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3" Jul 06 19:30:33.636810 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.636788 2561 scope.go:117] "RemoveContainer" containerID="db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd" Jul 06 19:30:33.643644 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.643626 2561 scope.go:117] "RemoveContainer" containerID="668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960" Jul 06 19:30:33.643891 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:33.643872 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960\": container with ID starting with 668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960 not found: ID does not exist" containerID="668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960" Jul 06 19:30:33.643937 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.643901 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960"} err="failed to get container status \"668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960\": rpc error: code = NotFound desc = could not find container \"668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960\": container with ID starting with 668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960 not found: ID does not exist" Jul 06 19:30:33.643937 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.643919 2561 scope.go:117] "RemoveContainer" containerID="7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb" Jul 06 19:30:33.644157 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:33.644142 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb\": container with ID starting with 7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb not found: ID does not exist" containerID="7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb" Jul 06 19:30:33.644205 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.644160 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb"} err="failed to get container status \"7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb\": rpc error: code = NotFound desc = could not find container \"7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb\": container with ID starting with 7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb not found: ID does not exist" Jul 06 19:30:33.644205 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.644178 2561 scope.go:117] "RemoveContainer" containerID="f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae" Jul 06 19:30:33.644466 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:33.644445 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae\": container with ID starting with f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae not found: ID does not exist" containerID="f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae" Jul 06 19:30:33.644527 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.644470 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae"} err="failed to get container status \"f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae\": rpc error: code = NotFound desc = could not find container \"f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae\": container with ID starting with f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae not found: ID does not exist" Jul 06 19:30:33.644527 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.644485 2561 scope.go:117] "RemoveContainer" containerID="7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022" Jul 06 19:30:33.644688 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:33.644673 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022\": container with ID starting with 7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022 not found: ID does not exist" containerID="7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022" Jul 06 19:30:33.644725 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.644690 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022"} err="failed to get container status \"7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022\": rpc error: code = NotFound desc = could not find container \"7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022\": container with ID starting with 7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022 not found: ID does not exist" Jul 06 19:30:33.644725 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.644704 2561 scope.go:117] "RemoveContainer" containerID="ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1" Jul 06 19:30:33.644939 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:33.644922 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1\": container with ID starting with ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1 not found: ID does not exist" containerID="ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1" Jul 06 19:30:33.644984 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.644944 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1"} err="failed to get container status \"ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1\": rpc error: code = NotFound desc = could not find container \"ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1\": container with ID starting with ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1 not found: ID does not exist" Jul 06 19:30:33.644984 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.644959 2561 scope.go:117] "RemoveContainer" containerID="3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3" Jul 06 19:30:33.645172 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:33.645159 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3\": container with ID starting with 3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3 not found: ID does not exist" containerID="3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3" Jul 06 19:30:33.645205 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.645177 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3"} err="failed to get container status \"3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3\": rpc error: code = NotFound desc = could not find container \"3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3\": container with ID starting with 3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3 not found: ID does not exist" Jul 06 19:30:33.645205 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.645190 2561 scope.go:117] "RemoveContainer" containerID="db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd" Jul 06 19:30:33.645368 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:33.645351 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd\": container with ID starting with db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd not found: ID does not exist" containerID="db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd" Jul 06 19:30:33.645412 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.645372 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd"} err="failed to get container status \"db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd\": rpc error: code = NotFound desc = could not find container \"db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd\": container with ID starting with db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd not found: ID does not exist" Jul 06 19:30:33.645412 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.645383 2561 scope.go:117] "RemoveContainer" containerID="668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960" Jul 06 19:30:33.645588 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.645572 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960"} err="failed to get container status \"668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960\": rpc error: code = NotFound desc = could not find container \"668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960\": container with ID starting with 668587e8f3ec7bba8d86410b1712887fa6782e20f06217d135408975d90ce960 not found: ID does not exist" Jul 06 19:30:33.645639 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.645587 2561 scope.go:117] "RemoveContainer" containerID="7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb" Jul 06 19:30:33.645775 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.645759 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb"} err="failed to get container status \"7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb\": rpc error: code = NotFound desc = could not find container \"7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb\": container with ID starting with 7799a8726591cc4231e3dfa41320c85a3e670f113e8184d4166a587727b7fdeb not found: ID does not exist" Jul 06 19:30:33.645815 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.645776 2561 scope.go:117] "RemoveContainer" containerID="f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae" Jul 06 19:30:33.645970 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.645954 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae"} err="failed to get container status \"f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae\": rpc error: code = NotFound desc = could not find container \"f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae\": container with ID starting with f76ca5e2d7f0d41c373d41c74b6daa3c65f79a8f67562d5d4c699ceedeb119ae not found: ID does not exist" Jul 06 19:30:33.645970 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.645968 2561 scope.go:117] "RemoveContainer" containerID="7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022" Jul 06 19:30:33.646138 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.646124 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022"} err="failed to get container status \"7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022\": rpc error: code = NotFound desc = could not find container \"7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022\": container with ID starting with 7c3b8f2fde8ade86ec663110ad4b5c5dbb4da462f0b3276a6405688402ebe022 not found: ID does not exist" Jul 06 19:30:33.646188 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.646138 2561 scope.go:117] "RemoveContainer" containerID="ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1" Jul 06 19:30:33.646340 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.646326 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1"} err="failed to get container status \"ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1\": rpc error: code = NotFound desc = could not find container \"ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1\": container with ID starting with ba528f76f94bfb08c9ee81207dad3e074b27b57621d8bdbb86524cd08ed65ae1 not found: ID does not exist" Jul 06 19:30:33.646340 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.646339 2561 scope.go:117] "RemoveContainer" containerID="3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3" Jul 06 19:30:33.646521 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.646503 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3"} err="failed to get container status \"3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3\": rpc error: code = NotFound desc = could not find container \"3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3\": container with ID starting with 3aeb04bfa2d14ea35c3dec119a4a4f6f98768d006158a701221d6fa56bed07c3 not found: ID does not exist" Jul 06 19:30:33.646563 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.646522 2561 scope.go:117] "RemoveContainer" containerID="db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd" Jul 06 19:30:33.646735 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.646714 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd"} err="failed to get container status \"db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd\": rpc error: code = NotFound desc = could not find container \"db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd\": container with ID starting with db132c447643688702733418bf6ad22352714850259778045d5dbe87f4e8d3fd not found: ID does not exist" Jul 06 19:30:33.652737 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.652717 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:30:33.653000 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.652989 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="kube-rbac-proxy" Jul 06 19:30:33.653040 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653002 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="kube-rbac-proxy" Jul 06 19:30:33.653040 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653015 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="prom-label-proxy" Jul 06 19:30:33.653040 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653021 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="prom-label-proxy" Jul 06 19:30:33.653040 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653028 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8" containerName="console" Jul 06 19:30:33.653040 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653033 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8" containerName="console" Jul 06 19:30:33.653040 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653041 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="kube-rbac-proxy-web" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653046 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="kube-rbac-proxy-web" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653054 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="init-config-reloader" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653059 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="init-config-reloader" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653065 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="config-reloader" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653070 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="config-reloader" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653076 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="kube-rbac-proxy-metric" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653081 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="kube-rbac-proxy-metric" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653088 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c819dedf-7c04-4003-8555-fdb5e5857f22" containerName="console" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653093 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c819dedf-7c04-4003-8555-fdb5e5857f22" containerName="console" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653099 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="alertmanager" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653104 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="alertmanager" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653177 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c819dedf-7c04-4003-8555-fdb5e5857f22" containerName="console" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653189 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="alertmanager" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653199 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2fbcb00-8249-4c0a-a7f9-da60d04e7ad8" containerName="console" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653208 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="kube-rbac-proxy" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653218 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="prom-label-proxy" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653229 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="config-reloader" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653237 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="kube-rbac-proxy-metric" Jul 06 19:30:33.653341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.653246 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" containerName="kube-rbac-proxy-web" Jul 06 19:30:33.658281 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.658264 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.660809 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.660793 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 19:30:33.660951 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.660793 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 19:30:33.661054 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.661040 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 19:30:33.661054 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.661047 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 19:30:33.661211 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.661097 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 19:30:33.661211 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.661155 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 19:30:33.661318 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.661265 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 19:30:33.661318 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.661275 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 19:30:33.661447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.661335 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s6dss\"" Jul 06 19:30:33.666717 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.666700 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 19:30:33.674508 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.674486 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:30:33.686504 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.686484 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2095283-08a9-4fc7-b28f-46552ea28b04" path="/var/lib/kubelet/pods/a2095283-08a9-4fc7-b28f-46552ea28b04/volumes" Jul 06 19:30:33.766356 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.766325 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39c14ca0-db08-4cbe-b760-720e9c6210a1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.766356 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.766359 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xp7q\" (UniqueName: \"kubernetes.io/projected/39c14ca0-db08-4cbe-b760-720e9c6210a1-kube-api-access-9xp7q\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.766566 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.766383 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39c14ca0-db08-4cbe-b760-720e9c6210a1-config-out\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.766566 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.766399 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.766566 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.766470 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.766566 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.766511 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39c14ca0-db08-4cbe-b760-720e9c6210a1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.766566 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.766529 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-config-volume\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.766566 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.766544 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.766566 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.766559 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-web-config\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.766885 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.766594 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/39c14ca0-db08-4cbe-b760-720e9c6210a1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.766885 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.766635 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.766885 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.766654 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39c14ca0-db08-4cbe-b760-720e9c6210a1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.766885 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.766675 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.867771 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.867688 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/39c14ca0-db08-4cbe-b760-720e9c6210a1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.867771 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.867727 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.867771 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.867753 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39c14ca0-db08-4cbe-b760-720e9c6210a1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.867771 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.867775 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.868090 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.867820 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39c14ca0-db08-4cbe-b760-720e9c6210a1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.868090 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.867845 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xp7q\" (UniqueName: \"kubernetes.io/projected/39c14ca0-db08-4cbe-b760-720e9c6210a1-kube-api-access-9xp7q\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.868090 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.867877 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39c14ca0-db08-4cbe-b760-720e9c6210a1-config-out\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.868090 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.867902 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.868090 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.867936 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.868090 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.867985 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39c14ca0-db08-4cbe-b760-720e9c6210a1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.868090 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.868011 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-config-volume\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.868090 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.868038 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.868090 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.868061 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-web-config\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.868563 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.868151 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/39c14ca0-db08-4cbe-b760-720e9c6210a1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.868620 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.868574 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39c14ca0-db08-4cbe-b760-720e9c6210a1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.868972 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.868876 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39c14ca0-db08-4cbe-b760-720e9c6210a1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.870954 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.870919 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.871074 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.870967 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.871074 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.871011 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.871250 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.871229 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39c14ca0-db08-4cbe-b760-720e9c6210a1-config-out\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.871529 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.871508 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-web-config\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.871597 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.871527 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.871597 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.871513 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-config-volume\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.871597 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.871575 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39c14ca0-db08-4cbe-b760-720e9c6210a1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.872312 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.872277 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39c14ca0-db08-4cbe-b760-720e9c6210a1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.876505 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.876489 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xp7q\" (UniqueName: \"kubernetes.io/projected/39c14ca0-db08-4cbe-b760-720e9c6210a1-kube-api-access-9xp7q\") pod \"alertmanager-main-0\" (UID: \"39c14ca0-db08-4cbe-b760-720e9c6210a1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:33.968842 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:33.968803 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:30:34.228525 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:34.228495 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:30:34.601575 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:34.601475 2561 generic.go:356] "Generic (PLEG): container finished" podID="39c14ca0-db08-4cbe-b760-720e9c6210a1" containerID="090004a219a7defd16c3f8bbd32a573b9a05d862009ab396e60991f00b6d441c" exitCode=0 Jul 06 19:30:34.601575 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:34.601526 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39c14ca0-db08-4cbe-b760-720e9c6210a1","Type":"ContainerDied","Data":"090004a219a7defd16c3f8bbd32a573b9a05d862009ab396e60991f00b6d441c"} Jul 06 19:30:34.602067 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:34.601597 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39c14ca0-db08-4cbe-b760-720e9c6210a1","Type":"ContainerStarted","Data":"28aa62397d41c8bb9c68a26d1fcebcad00ab2e95d490c968f0191d8a50b55248"} Jul 06 19:30:35.609578 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:35.609543 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39c14ca0-db08-4cbe-b760-720e9c6210a1","Type":"ContainerStarted","Data":"50a8ae9f7e6ed31b59ad2b3df76dc96c1887706a639fb6e4f5f62a0bd2795c41"} Jul 06 19:30:35.610028 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:35.609618 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39c14ca0-db08-4cbe-b760-720e9c6210a1","Type":"ContainerStarted","Data":"818af0be262d55ca83404fca11f9cac1594ffc205010365e6bd6b7e545d59af5"} Jul 06 19:30:35.610028 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:35.609659 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39c14ca0-db08-4cbe-b760-720e9c6210a1","Type":"ContainerStarted","Data":"322996c8d5bef431e2eb8d42b538c8962c9acbcff26922435ae61a6ebcddf6ab"} Jul 06 19:30:35.610028 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:35.609708 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39c14ca0-db08-4cbe-b760-720e9c6210a1","Type":"ContainerStarted","Data":"2e2708c9cc3c76d93b796da0479099814092da9764f75a8a0d1b39104e032c74"} Jul 06 19:30:35.610028 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:35.609748 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39c14ca0-db08-4cbe-b760-720e9c6210a1","Type":"ContainerStarted","Data":"f97de2154144fb10b878fdb3dc6c0e3e909bc9403205de88d8669e09e01b1aad"} Jul 06 19:30:35.610028 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:35.609786 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39c14ca0-db08-4cbe-b760-720e9c6210a1","Type":"ContainerStarted","Data":"3842e5105d4c7cd3de99b775478fa048290df66ece66ecabc7bbaa808b5829fd"} Jul 06 19:30:35.637022 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:35.636963 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.622424755 podStartE2EDuration="2.636942849s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.518094ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:30:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:30:35.635245254 +0000 UTC m=+170.502920811" watchObservedRunningTime="2026-07-06 19:30:35.636942849 +0000 UTC m=+170.504618413" Jul 06 19:30:36.035772 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.035735 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-689b96b8dd-sz8r7"] Jul 06 19:30:36.039156 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.039129 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.041737 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.041710 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 19:30:36.041876 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.041855 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 19:30:36.041961 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.041895 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 19:30:36.042211 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.042198 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 19:30:36.042263 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.042219 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-qllg4\"" Jul 06 19:30:36.042634 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.042618 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 19:30:36.053797 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.051905 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 19:30:36.056483 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.056458 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-689b96b8dd-sz8r7"] Jul 06 19:30:36.191226 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.191185 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/208218e1-8f44-408c-9e2f-a4484b7045f5-federate-client-tls\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.191226 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.191226 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/208218e1-8f44-408c-9e2f-a4484b7045f5-metrics-client-ca\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.191489 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.191263 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/208218e1-8f44-408c-9e2f-a4484b7045f5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.191489 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.191394 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/208218e1-8f44-408c-9e2f-a4484b7045f5-secret-telemeter-client\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.191489 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.191435 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/208218e1-8f44-408c-9e2f-a4484b7045f5-serving-certs-ca-bundle\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.191489 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.191462 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/208218e1-8f44-408c-9e2f-a4484b7045f5-telemeter-client-tls\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.191614 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.191491 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkrqv\" (UniqueName: \"kubernetes.io/projected/208218e1-8f44-408c-9e2f-a4484b7045f5-kube-api-access-nkrqv\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.191614 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.191572 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/208218e1-8f44-408c-9e2f-a4484b7045f5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.292376 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.292242 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/208218e1-8f44-408c-9e2f-a4484b7045f5-secret-telemeter-client\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.292376 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.292319 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/208218e1-8f44-408c-9e2f-a4484b7045f5-serving-certs-ca-bundle\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.292376 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.292346 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/208218e1-8f44-408c-9e2f-a4484b7045f5-telemeter-client-tls\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.292376 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.292370 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkrqv\" (UniqueName: \"kubernetes.io/projected/208218e1-8f44-408c-9e2f-a4484b7045f5-kube-api-access-nkrqv\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.292783 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.292411 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/208218e1-8f44-408c-9e2f-a4484b7045f5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.292783 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.292447 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/208218e1-8f44-408c-9e2f-a4484b7045f5-federate-client-tls\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.292783 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.292474 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/208218e1-8f44-408c-9e2f-a4484b7045f5-metrics-client-ca\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.292783 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.292526 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/208218e1-8f44-408c-9e2f-a4484b7045f5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.293285 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.293250 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/208218e1-8f44-408c-9e2f-a4484b7045f5-serving-certs-ca-bundle\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.293511 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.293250 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/208218e1-8f44-408c-9e2f-a4484b7045f5-metrics-client-ca\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.293511 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.293442 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/208218e1-8f44-408c-9e2f-a4484b7045f5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.295113 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.295087 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/208218e1-8f44-408c-9e2f-a4484b7045f5-secret-telemeter-client\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.295222 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.295172 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/208218e1-8f44-408c-9e2f-a4484b7045f5-federate-client-tls\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.295365 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.295342 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/208218e1-8f44-408c-9e2f-a4484b7045f5-telemeter-client-tls\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.295469 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.295445 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/208218e1-8f44-408c-9e2f-a4484b7045f5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.301514 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.301485 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkrqv\" (UniqueName: \"kubernetes.io/projected/208218e1-8f44-408c-9e2f-a4484b7045f5-kube-api-access-nkrqv\") pod \"telemeter-client-689b96b8dd-sz8r7\" (UID: \"208218e1-8f44-408c-9e2f-a4484b7045f5\") " pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.344128 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.344091 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:30:36.344420 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.344389 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="prometheus" containerID="cri-o://cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb" gracePeriod=600 Jul 06 19:30:36.344558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.344392 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="kube-rbac-proxy" containerID="cri-o://19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c" gracePeriod=600 Jul 06 19:30:36.344558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.344438 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="kube-rbac-proxy-thanos" containerID="cri-o://fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e" gracePeriod=600 Jul 06 19:30:36.344558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.344482 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="config-reloader" containerID="cri-o://d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9" gracePeriod=600 Jul 06 19:30:36.344558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.344446 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="kube-rbac-proxy-web" containerID="cri-o://5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302" gracePeriod=600 Jul 06 19:30:36.344558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.344447 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="thanos-sidecar" containerID="cri-o://61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff" gracePeriod=600 Jul 06 19:30:36.356238 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.356120 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" Jul 06 19:30:36.536970 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.536906 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-689b96b8dd-sz8r7"] Jul 06 19:30:36.539064 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:30:36.539032 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod208218e1_8f44_408c_9e2f_a4484b7045f5.slice/crio-3f9c2edf3467b7ddc237a2305023bbb71f4c29f2b9dfb7d8a996c19d10c4e6ee WatchSource:0}: Error finding container 3f9c2edf3467b7ddc237a2305023bbb71f4c29f2b9dfb7d8a996c19d10c4e6ee: Status 404 returned error can't find the container with id 3f9c2edf3467b7ddc237a2305023bbb71f4c29f2b9dfb7d8a996c19d10c4e6ee Jul 06 19:30:36.606306 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.606255 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:36.615766 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.615731 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" event={"ID":"208218e1-8f44-408c-9e2f-a4484b7045f5","Type":"ContainerStarted","Data":"3f9c2edf3467b7ddc237a2305023bbb71f4c29f2b9dfb7d8a996c19d10c4e6ee"} Jul 06 19:30:36.619075 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619048 2561 generic.go:356] "Generic (PLEG): container finished" podID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerID="fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e" exitCode=0 Jul 06 19:30:36.619075 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619070 2561 generic.go:356] "Generic (PLEG): container finished" podID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerID="19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c" exitCode=0 Jul 06 19:30:36.619075 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619076 2561 generic.go:356] "Generic (PLEG): container finished" podID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerID="5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302" exitCode=0 Jul 06 19:30:36.619075 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619082 2561 generic.go:356] "Generic (PLEG): container finished" podID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerID="61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff" exitCode=0 Jul 06 19:30:36.619075 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619088 2561 generic.go:356] "Generic (PLEG): container finished" podID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerID="d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9" exitCode=0 Jul 06 19:30:36.619075 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619094 2561 generic.go:356] "Generic (PLEG): container finished" podID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerID="cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb" exitCode=0 Jul 06 19:30:36.619558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619119 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerDied","Data":"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e"} Jul 06 19:30:36.619558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619176 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerDied","Data":"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c"} Jul 06 19:30:36.619558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619122 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:36.619558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619206 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerDied","Data":"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302"} Jul 06 19:30:36.619558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619231 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerDied","Data":"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff"} Jul 06 19:30:36.619558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619254 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerDied","Data":"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9"} Jul 06 19:30:36.619558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619262 2561 scope.go:117] "RemoveContainer" containerID="fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e" Jul 06 19:30:36.619558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619278 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerDied","Data":"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb"} Jul 06 19:30:36.619558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.619328 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0d77e7a2-b170-4ef7-a1a5-b05037522b1f","Type":"ContainerDied","Data":"c65611a28b7de8874ad5bcfbbab9e722a200ef4492973730569bdcee5c23cc6a"} Jul 06 19:30:36.628973 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.628952 2561 scope.go:117] "RemoveContainer" containerID="19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c" Jul 06 19:30:36.638614 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.638592 2561 scope.go:117] "RemoveContainer" containerID="5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302" Jul 06 19:30:36.649461 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.649029 2561 scope.go:117] "RemoveContainer" containerID="61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff" Jul 06 19:30:36.659927 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.659896 2561 scope.go:117] "RemoveContainer" containerID="d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9" Jul 06 19:30:36.668517 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.668495 2561 scope.go:117] "RemoveContainer" containerID="cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb" Jul 06 19:30:36.679034 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.679013 2561 scope.go:117] "RemoveContainer" containerID="0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328" Jul 06 19:30:36.690732 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.690708 2561 scope.go:117] "RemoveContainer" containerID="fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e" Jul 06 19:30:36.691075 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:36.691056 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e\": container with ID starting with fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e not found: ID does not exist" containerID="fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e" Jul 06 19:30:36.691183 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.691086 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e"} err="failed to get container status \"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e\": rpc error: code = NotFound desc = could not find container \"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e\": container with ID starting with fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e not found: ID does not exist" Jul 06 19:30:36.691183 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.691109 2561 scope.go:117] "RemoveContainer" containerID="19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c" Jul 06 19:30:36.691479 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:36.691458 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c\": container with ID starting with 19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c not found: ID does not exist" containerID="19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c" Jul 06 19:30:36.691534 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.691488 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c"} err="failed to get container status \"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c\": rpc error: code = NotFound desc = could not find container \"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c\": container with ID starting with 19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c not found: ID does not exist" Jul 06 19:30:36.691534 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.691510 2561 scope.go:117] "RemoveContainer" containerID="5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302" Jul 06 19:30:36.691867 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:36.691839 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302\": container with ID starting with 5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302 not found: ID does not exist" containerID="5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302" Jul 06 19:30:36.691983 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.691874 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302"} err="failed to get container status \"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302\": rpc error: code = NotFound desc = could not find container \"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302\": container with ID starting with 5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302 not found: ID does not exist" Jul 06 19:30:36.691983 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.691897 2561 scope.go:117] "RemoveContainer" containerID="61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff" Jul 06 19:30:36.692189 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:36.692164 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff\": container with ID starting with 61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff not found: ID does not exist" containerID="61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff" Jul 06 19:30:36.692262 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.692200 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff"} err="failed to get container status \"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff\": rpc error: code = NotFound desc = could not find container \"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff\": container with ID starting with 61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff not found: ID does not exist" Jul 06 19:30:36.692262 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.692219 2561 scope.go:117] "RemoveContainer" containerID="d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9" Jul 06 19:30:36.692523 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:36.692493 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9\": container with ID starting with d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9 not found: ID does not exist" containerID="d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9" Jul 06 19:30:36.692607 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.692521 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9"} err="failed to get container status \"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9\": rpc error: code = NotFound desc = could not find container \"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9\": container with ID starting with d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9 not found: ID does not exist" Jul 06 19:30:36.692607 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.692542 2561 scope.go:117] "RemoveContainer" containerID="cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb" Jul 06 19:30:36.692837 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:36.692786 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb\": container with ID starting with cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb not found: ID does not exist" containerID="cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb" Jul 06 19:30:36.692837 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.692810 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb"} err="failed to get container status \"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb\": rpc error: code = NotFound desc = could not find container \"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb\": container with ID starting with cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb not found: ID does not exist" Jul 06 19:30:36.692837 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.692830 2561 scope.go:117] "RemoveContainer" containerID="0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328" Jul 06 19:30:36.693136 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:30:36.693090 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328\": container with ID starting with 0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328 not found: ID does not exist" containerID="0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328" Jul 06 19:30:36.693136 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.693118 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328"} err="failed to get container status \"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328\": rpc error: code = NotFound desc = could not find container \"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328\": container with ID starting with 0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328 not found: ID does not exist" Jul 06 19:30:36.693275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.693141 2561 scope.go:117] "RemoveContainer" containerID="fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e" Jul 06 19:30:36.693451 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.693413 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e"} err="failed to get container status \"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e\": rpc error: code = NotFound desc = could not find container \"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e\": container with ID starting with fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e not found: ID does not exist" Jul 06 19:30:36.693451 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.693437 2561 scope.go:117] "RemoveContainer" containerID="19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c" Jul 06 19:30:36.693712 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.693691 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c"} err="failed to get container status \"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c\": rpc error: code = NotFound desc = could not find container \"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c\": container with ID starting with 19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c not found: ID does not exist" Jul 06 19:30:36.693788 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.693714 2561 scope.go:117] "RemoveContainer" containerID="5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302" Jul 06 19:30:36.694007 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.693970 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302"} err="failed to get container status \"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302\": rpc error: code = NotFound desc = could not find container \"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302\": container with ID starting with 5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302 not found: ID does not exist" Jul 06 19:30:36.694109 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.694009 2561 scope.go:117] "RemoveContainer" containerID="61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff" Jul 06 19:30:36.694497 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.694413 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff"} err="failed to get container status \"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff\": rpc error: code = NotFound desc = could not find container \"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff\": container with ID starting with 61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff not found: ID does not exist" Jul 06 19:30:36.694497 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.694437 2561 scope.go:117] "RemoveContainer" containerID="d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9" Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695107 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-web-config\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695145 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-tls-assets\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695160 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-config\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695156 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9"} err="failed to get container status \"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9\": rpc error: code = NotFound desc = could not find container \"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9\": container with ID starting with d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9 not found: ID does not exist" Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695180 2561 scope.go:117] "RemoveContainer" containerID="cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb" Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695189 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-db\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695214 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-thanos-prometheus-http-client-file\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695248 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-tls\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695280 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695326 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-2\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695358 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-trusted-ca-bundle\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695385 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-1\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695423 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-metrics-client-ca\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695452 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695476 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-grpc-tls\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695516 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-0\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695544 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-config-out\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.695608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695597 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-kubelet-serving-ca-bundle\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.696546 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695628 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-serving-certs-ca-bundle\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.696546 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695694 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-metrics-client-certs\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.696546 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695720 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-kube-rbac-proxy\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.696546 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695753 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2c72d\" (UniqueName: \"kubernetes.io/projected/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-kube-api-access-2c72d\") pod \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\" (UID: \"0d77e7a2-b170-4ef7-a1a5-b05037522b1f\") " Jul 06 19:30:36.696546 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.695814 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:36.696546 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.696058 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.696546 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.696494 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb"} err="failed to get container status \"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb\": rpc error: code = NotFound desc = could not find container \"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb\": container with ID starting with cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb not found: ID does not exist" Jul 06 19:30:36.696546 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.696522 2561 scope.go:117] "RemoveContainer" containerID="0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328" Jul 06 19:30:36.696945 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.696633 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:30:36.698345 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.698320 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:36.698590 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.698564 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-kube-api-access-2c72d" (OuterVolumeSpecName: "kube-api-access-2c72d") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "kube-api-access-2c72d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:30:36.698668 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.698643 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:36.699084 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.699059 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:36.699163 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.699084 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328"} err="failed to get container status \"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328\": rpc error: code = NotFound desc = could not find container \"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328\": container with ID starting with 0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328 not found: ID does not exist" Jul 06 19:30:36.699163 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.699111 2561 scope.go:117] "RemoveContainer" containerID="fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e" Jul 06 19:30:36.699565 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.699260 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:36.699636 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.699561 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:36.699972 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.699731 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e"} err="failed to get container status \"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e\": rpc error: code = NotFound desc = could not find container \"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e\": container with ID starting with fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e not found: ID does not exist" Jul 06 19:30:36.699972 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.699757 2561 scope.go:117] "RemoveContainer" containerID="19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c" Jul 06 19:30:36.699972 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.699870 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:36.700606 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.700175 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c"} err="failed to get container status \"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c\": rpc error: code = NotFound desc = could not find container \"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c\": container with ID starting with 19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c not found: ID does not exist" Jul 06 19:30:36.700606 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.700211 2561 scope.go:117] "RemoveContainer" containerID="5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302" Jul 06 19:30:36.700606 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.700427 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-config-out" (OuterVolumeSpecName: "config-out") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:30:36.700606 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.700490 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302"} err="failed to get container status \"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302\": rpc error: code = NotFound desc = could not find container \"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302\": container with ID starting with 5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302 not found: ID does not exist" Jul 06 19:30:36.700606 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.700512 2561 scope.go:117] "RemoveContainer" containerID="61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff" Jul 06 19:30:36.700606 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.700572 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:30:36.700606 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.700562 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-config" (OuterVolumeSpecName: "config") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:36.701114 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.701049 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:36.701198 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.701119 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:36.701710 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.701681 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff"} err="failed to get container status \"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff\": rpc error: code = NotFound desc = could not find container \"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff\": container with ID starting with 61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff not found: ID does not exist" Jul 06 19:30:36.701795 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.701711 2561 scope.go:117] "RemoveContainer" containerID="d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9" Jul 06 19:30:36.701934 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.701911 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:36.702067 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.702031 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9"} err="failed to get container status \"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9\": rpc error: code = NotFound desc = could not find container \"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9\": container with ID starting with d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9 not found: ID does not exist" Jul 06 19:30:36.702140 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.702071 2561 scope.go:117] "RemoveContainer" containerID="cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb" Jul 06 19:30:36.702244 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.702199 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:30:36.702405 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.702376 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb"} err="failed to get container status \"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb\": rpc error: code = NotFound desc = could not find container \"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb\": container with ID starting with cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb not found: ID does not exist" Jul 06 19:30:36.702405 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.702406 2561 scope.go:117] "RemoveContainer" containerID="0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328" Jul 06 19:30:36.702734 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.702707 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328"} err="failed to get container status \"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328\": rpc error: code = NotFound desc = could not find container \"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328\": container with ID starting with 0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328 not found: ID does not exist" Jul 06 19:30:36.702819 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.702739 2561 scope.go:117] "RemoveContainer" containerID="fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e" Jul 06 19:30:36.702819 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.702786 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:36.702941 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.702922 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:36.703044 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.703024 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e"} err="failed to get container status \"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e\": rpc error: code = NotFound desc = could not find container \"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e\": container with ID starting with fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e not found: ID does not exist" Jul 06 19:30:36.703114 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.703045 2561 scope.go:117] "RemoveContainer" containerID="19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c" Jul 06 19:30:36.703365 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.703313 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c"} err="failed to get container status \"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c\": rpc error: code = NotFound desc = could not find container \"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c\": container with ID starting with 19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c not found: ID does not exist" Jul 06 19:30:36.703365 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.703341 2561 scope.go:117] "RemoveContainer" containerID="5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302" Jul 06 19:30:36.703504 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.703412 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:36.703604 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.703579 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302"} err="failed to get container status \"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302\": rpc error: code = NotFound desc = could not find container \"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302\": container with ID starting with 5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302 not found: ID does not exist" Jul 06 19:30:36.703668 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.703608 2561 scope.go:117] "RemoveContainer" containerID="61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff" Jul 06 19:30:36.703891 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.703870 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff"} err="failed to get container status \"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff\": rpc error: code = NotFound desc = could not find container \"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff\": container with ID starting with 61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff not found: ID does not exist" Jul 06 19:30:36.703954 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.703892 2561 scope.go:117] "RemoveContainer" containerID="d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9" Jul 06 19:30:36.704173 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.704154 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9"} err="failed to get container status \"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9\": rpc error: code = NotFound desc = could not find container \"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9\": container with ID starting with d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9 not found: ID does not exist" Jul 06 19:30:36.704236 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.704175 2561 scope.go:117] "RemoveContainer" containerID="cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb" Jul 06 19:30:36.704473 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.704454 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb"} err="failed to get container status \"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb\": rpc error: code = NotFound desc = could not find container \"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb\": container with ID starting with cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb not found: ID does not exist" Jul 06 19:30:36.704550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.704473 2561 scope.go:117] "RemoveContainer" containerID="0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328" Jul 06 19:30:36.704743 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.704720 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328"} err="failed to get container status \"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328\": rpc error: code = NotFound desc = could not find container \"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328\": container with ID starting with 0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328 not found: ID does not exist" Jul 06 19:30:36.704792 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.704745 2561 scope.go:117] "RemoveContainer" containerID="fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e" Jul 06 19:30:36.705015 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.704994 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e"} err="failed to get container status \"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e\": rpc error: code = NotFound desc = could not find container \"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e\": container with ID starting with fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e not found: ID does not exist" Jul 06 19:30:36.705091 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.705018 2561 scope.go:117] "RemoveContainer" containerID="19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c" Jul 06 19:30:36.705274 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.705251 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c"} err="failed to get container status \"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c\": rpc error: code = NotFound desc = could not find container \"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c\": container with ID starting with 19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c not found: ID does not exist" Jul 06 19:30:36.705443 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.705275 2561 scope.go:117] "RemoveContainer" containerID="5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.705629 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302"} err="failed to get container status \"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302\": rpc error: code = NotFound desc = could not find container \"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302\": container with ID starting with 5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302 not found: ID does not exist" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.705657 2561 scope.go:117] "RemoveContainer" containerID="61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.705871 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff"} err="failed to get container status \"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff\": rpc error: code = NotFound desc = could not find container \"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff\": container with ID starting with 61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff not found: ID does not exist" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.705889 2561 scope.go:117] "RemoveContainer" containerID="d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.706099 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9"} err="failed to get container status \"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9\": rpc error: code = NotFound desc = could not find container \"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9\": container with ID starting with d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9 not found: ID does not exist" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.706118 2561 scope.go:117] "RemoveContainer" containerID="cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.706534 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb"} err="failed to get container status \"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb\": rpc error: code = NotFound desc = could not find container \"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb\": container with ID starting with cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb not found: ID does not exist" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.706554 2561 scope.go:117] "RemoveContainer" containerID="0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.706765 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328"} err="failed to get container status \"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328\": rpc error: code = NotFound desc = could not find container \"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328\": container with ID starting with 0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328 not found: ID does not exist" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.706783 2561 scope.go:117] "RemoveContainer" containerID="fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.706977 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e"} err="failed to get container status \"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e\": rpc error: code = NotFound desc = could not find container \"fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e\": container with ID starting with fd0b84c2c60648f0f90baeac859ea63e9b3da4bcf6a4ec38070d46466598cb2e not found: ID does not exist" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.706994 2561 scope.go:117] "RemoveContainer" containerID="19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.707181 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c"} err="failed to get container status \"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c\": rpc error: code = NotFound desc = could not find container \"19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c\": container with ID starting with 19455342b9ca12c9fa22f3225d754b90f301e59008a1cf5929044942ffb0804c not found: ID does not exist" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.707199 2561 scope.go:117] "RemoveContainer" containerID="5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.707415 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302"} err="failed to get container status \"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302\": rpc error: code = NotFound desc = could not find container \"5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302\": container with ID starting with 5991890c9756563493395d26bf3d2407a2f153e8ba117699cd3a472c1d65c302 not found: ID does not exist" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.707433 2561 scope.go:117] "RemoveContainer" containerID="61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff" Jul 06 19:30:36.708185 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.707643 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff"} err="failed to get container status \"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff\": rpc error: code = NotFound desc = could not find container \"61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff\": container with ID starting with 61b491b6a46f8e9db8631720013ea6ecab91ee9edb7624d83c235ba5aeb856ff not found: ID does not exist" Jul 06 19:30:36.709054 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.707661 2561 scope.go:117] "RemoveContainer" containerID="d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9" Jul 06 19:30:36.709054 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.708343 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9"} err="failed to get container status \"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9\": rpc error: code = NotFound desc = could not find container \"d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9\": container with ID starting with d7bdbcde13602bcd37c607812ef8f8127595dbac45baca472d60d9d8c7501bb9 not found: ID does not exist" Jul 06 19:30:36.709054 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.708360 2561 scope.go:117] "RemoveContainer" containerID="cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb" Jul 06 19:30:36.709054 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.708584 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb"} err="failed to get container status \"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb\": rpc error: code = NotFound desc = could not find container \"cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb\": container with ID starting with cf19b55490ba4134bd09e4d620ccb48dfd1e569f7b2d5c0cc0aaa66d136a22cb not found: ID does not exist" Jul 06 19:30:36.709054 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.708600 2561 scope.go:117] "RemoveContainer" containerID="0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328" Jul 06 19:30:36.709054 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.708834 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328"} err="failed to get container status \"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328\": rpc error: code = NotFound desc = could not find container \"0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328\": container with ID starting with 0fd0d99e05d38d56ac6ecb5ffb69c64a419159e874deb7dbd5c73aa4e1b1a328 not found: ID does not exist" Jul 06 19:30:36.714454 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.714428 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-web-config" (OuterVolumeSpecName: "web-config") pod "0d77e7a2-b170-4ef7-a1a5-b05037522b1f" (UID: "0d77e7a2-b170-4ef7-a1a5-b05037522b1f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796741 2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-web-config\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796777 2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-tls-assets\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796790 2561 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-config\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796804 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-db\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796817 2561 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796830 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796844 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796857 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796873 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796885 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-metrics-client-ca\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796903 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796916 2561 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-grpc-tls\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796930 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796942 2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-config-out\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796958 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796970 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796983 2561 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-metrics-client-certs\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.796994 2561 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-secret-kube-rbac-proxy\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.797991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.797009 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c72d\" (UniqueName: \"kubernetes.io/projected/0d77e7a2-b170-4ef7-a1a5-b05037522b1f-kube-api-access-2c72d\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:30:36.945548 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.945517 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:30:36.953528 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.953502 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:30:36.983286 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983258 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:30:36.983597 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983583 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="init-config-reloader" Jul 06 19:30:36.983639 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983599 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="init-config-reloader" Jul 06 19:30:36.983639 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983608 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="prometheus" Jul 06 19:30:36.983639 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983613 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="prometheus" Jul 06 19:30:36.983639 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983619 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="kube-rbac-proxy" Jul 06 19:30:36.983639 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983625 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="kube-rbac-proxy" Jul 06 19:30:36.983780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983637 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="kube-rbac-proxy-thanos" Jul 06 19:30:36.983780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983664 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="kube-rbac-proxy-thanos" Jul 06 19:30:36.983780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983673 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="thanos-sidecar" Jul 06 19:30:36.983780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983679 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="thanos-sidecar" Jul 06 19:30:36.983780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983686 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="config-reloader" Jul 06 19:30:36.983780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983691 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="config-reloader" Jul 06 19:30:36.983780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983698 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="kube-rbac-proxy-web" Jul 06 19:30:36.983780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983702 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="kube-rbac-proxy-web" Jul 06 19:30:36.983780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983750 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="thanos-sidecar" Jul 06 19:30:36.983780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983759 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="config-reloader" Jul 06 19:30:36.983780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983765 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="kube-rbac-proxy-web" Jul 06 19:30:36.983780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983771 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="kube-rbac-proxy-thanos" Jul 06 19:30:36.983780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983777 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="prometheus" Jul 06 19:30:36.983780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.983783 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" containerName="kube-rbac-proxy" Jul 06 19:30:36.989017 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.988994 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:36.991325 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.991282 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 19:30:36.991516 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.991499 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 19:30:36.991618 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.991565 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fkkn5\"" Jul 06 19:30:36.991618 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.991579 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 19:30:36.992094 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.992075 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 19:30:36.992243 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.992200 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 19:30:36.992682 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.992662 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 19:30:36.993082 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.993060 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 19:30:36.993082 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.993076 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 19:30:36.993223 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.993076 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 19:30:36.993223 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.993116 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 19:30:36.993440 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.993421 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 19:30:36.993531 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.993448 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7pjbf692hlv4\"" Jul 06 19:30:36.993704 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.993687 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 19:30:36.996041 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.996019 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 19:30:36.999427 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:36.999386 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 19:30:37.008669 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.008639 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:30:37.099193 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099102 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-config\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099193 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099150 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099197 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-config-out\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099224 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099249 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099266 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099286 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7kw4\" (UniqueName: \"kubernetes.io/projected/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-kube-api-access-d7kw4\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099388 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-web-config\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099406 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099423 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099439 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099460 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099492 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099520 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099563 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099586 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099628 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099663 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099683 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.099789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.099716 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.201476 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-config-out\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.201523 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.201710 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.201760 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.201794 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7kw4\" (UniqueName: \"kubernetes.io/projected/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-kube-api-access-d7kw4\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.201848 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-web-config\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.201880 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.201913 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.201946 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.201977 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.202006 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.202035 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.202100 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.202131 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.202156 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.202198 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.206926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.202233 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.207884 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.202273 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.207884 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.202335 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-config\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.207884 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.202398 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.207884 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.202891 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.207884 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.203551 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.207884 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.207607 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.210661 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.210134 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.210661 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.210231 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.210661 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.210282 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.210661 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.210370 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-web-config\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.210661 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.210426 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.210992 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.210702 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.211275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.211203 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.211275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.211210 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.211275 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.211255 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.212455 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.212429 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.213749 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.213699 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-config-out\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.214372 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.213947 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.214372 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.214279 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.214587 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.214481 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.214748 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.214727 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.214953 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.214932 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-config\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.216162 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.216144 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7kw4\" (UniqueName: \"kubernetes.io/projected/f88086f4-aa96-41dc-9ac4-a32d5af1c4a8-kube-api-access-d7kw4\") pod \"prometheus-k8s-0\" (UID: \"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.299668 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.299589 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:37.472358 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.472194 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:30:37.474746 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:30:37.474717 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf88086f4_aa96_41dc_9ac4_a32d5af1c4a8.slice/crio-eca2e908bb10c51072385930316f266856382d107efc3b1c6e42ef6efe00955b WatchSource:0}: Error finding container eca2e908bb10c51072385930316f266856382d107efc3b1c6e42ef6efe00955b: Status 404 returned error can't find the container with id eca2e908bb10c51072385930316f266856382d107efc3b1c6e42ef6efe00955b Jul 06 19:30:37.626975 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.626902 2561 generic.go:356] "Generic (PLEG): container finished" podID="f88086f4-aa96-41dc-9ac4-a32d5af1c4a8" containerID="e3e13fe85fcbead4c0e34ff92c4695b5950b48a7c620356668e2966497ae3f82" exitCode=0 Jul 06 19:30:37.626975 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.626940 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8","Type":"ContainerDied","Data":"e3e13fe85fcbead4c0e34ff92c4695b5950b48a7c620356668e2966497ae3f82"} Jul 06 19:30:37.627467 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.626989 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8","Type":"ContainerStarted","Data":"eca2e908bb10c51072385930316f266856382d107efc3b1c6e42ef6efe00955b"} Jul 06 19:30:37.687141 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:37.687116 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d77e7a2-b170-4ef7-a1a5-b05037522b1f" path="/var/lib/kubelet/pods/0d77e7a2-b170-4ef7-a1a5-b05037522b1f/volumes" Jul 06 19:30:38.634537 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:38.634508 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" event={"ID":"208218e1-8f44-408c-9e2f-a4484b7045f5","Type":"ContainerStarted","Data":"33f5cb2d76a0b436a09b80cc498f2538e52822a5a0300356a0d567d34359fc6e"} Jul 06 19:30:38.638306 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:38.638264 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8","Type":"ContainerStarted","Data":"8bbe6c7e637a6e2ff9b78d97f1af233cbbaf9e3e317d21b24201caca80027862"} Jul 06 19:30:38.638447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:38.638343 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8","Type":"ContainerStarted","Data":"6a1fe5ab69aac95fbe94482c5f5ca160bd6c4a2134664b4e27bce8abb57c587f"} Jul 06 19:30:38.638447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:38.638370 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8","Type":"ContainerStarted","Data":"7c23483e304695be223918ebcad01573eab6ca30edac88b7975f31ed1bd624d8"} Jul 06 19:30:38.638447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:38.638401 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8","Type":"ContainerStarted","Data":"729df96e97b4b0ff7e6db503b13bc145cb78e75c5ddfba80393cd02479c94842"} Jul 06 19:30:38.638447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:38.638434 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8","Type":"ContainerStarted","Data":"f09abc22507fb0c96d0234d5146682103afb4712f5baaf20be7674c5d4757edd"} Jul 06 19:30:38.638690 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:38.638469 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f88086f4-aa96-41dc-9ac4-a32d5af1c4a8","Type":"ContainerStarted","Data":"8667c6206154b2a8bd399f2b7a40afa3db9bdbd9ca214f4d2d2a9334d1eb3834"} Jul 06 19:30:39.644450 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:39.644414 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" event={"ID":"208218e1-8f44-408c-9e2f-a4484b7045f5","Type":"ContainerStarted","Data":"76dff6eb7bc1c69b8bc147b0677d22680210ed99d6f0c32db8a064c4fa88affc"} Jul 06 19:30:39.644906 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:39.644508 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" event={"ID":"208218e1-8f44-408c-9e2f-a4484b7045f5","Type":"ContainerStarted","Data":"1f6db16b313eece5f7d6b4cb241274996a3b695eb4d83d4c3fd0726be8f7d4b6"} Jul 06 19:30:39.670685 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:39.670633 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.64737789 podStartE2EDuration="3.670615459s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.237569ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:30:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:30:38.669100146 +0000 UTC m=+173.536775703" watchObservedRunningTime="2026-07-06 19:30:39.670615459 +0000 UTC m=+174.538291018" Jul 06 19:30:39.671486 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:39.671450 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-689b96b8dd-sz8r7" podStartSLOduration=1.686281025 podStartE2EDuration="3.67144106s" totalImagesPullingTime="1.985160035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:30:39.669520829 +0000 UTC m=+174.537196419" watchObservedRunningTime="2026-07-06 19:30:39.67144106 +0000 UTC m=+174.539116615" Jul 06 19:30:40.443425 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.443389 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f8bbd8b45-cl5kr"] Jul 06 19:30:40.447570 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.447543 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.458225 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.458198 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f8bbd8b45-cl5kr"] Jul 06 19:30:40.539095 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.539060 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-config\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.539095 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.539099 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-oauth-config\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.539268 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.539123 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-serving-cert\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.539268 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.539143 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-service-ca\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.539268 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.539174 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6zzv\" (UniqueName: \"kubernetes.io/projected/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-kube-api-access-z6zzv\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.539268 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.539208 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-trusted-ca-bundle\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.539268 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.539242 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-oauth-serving-cert\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.639868 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.639825 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6zzv\" (UniqueName: \"kubernetes.io/projected/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-kube-api-access-z6zzv\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.639868 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.639873 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-trusted-ca-bundle\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.640106 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.639900 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-oauth-serving-cert\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.640106 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.639945 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-config\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.640106 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.639968 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-oauth-config\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.640106 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.639997 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-serving-cert\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.640322 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.640155 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-service-ca\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.640794 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.640763 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-oauth-serving-cert\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.640904 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.640769 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-config\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.640904 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.640866 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-service-ca\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.641014 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.640958 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-trusted-ca-bundle\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.642473 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.642445 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-oauth-config\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.642576 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.642560 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-serving-cert\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.649671 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.649651 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6zzv\" (UniqueName: \"kubernetes.io/projected/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-kube-api-access-z6zzv\") pod \"console-7f8bbd8b45-cl5kr\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.759564 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.759470 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:40.897942 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:40.897914 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f8bbd8b45-cl5kr"] Jul 06 19:30:40.900127 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:30:40.900096 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67ebdf3b_f451_492f_aaf8_1e668ddfbde6.slice/crio-ae8219daa3ca57ac4944a40197155172c7f3f886457aea26a6b8b216e4018b21 WatchSource:0}: Error finding container ae8219daa3ca57ac4944a40197155172c7f3f886457aea26a6b8b216e4018b21: Status 404 returned error can't find the container with id ae8219daa3ca57ac4944a40197155172c7f3f886457aea26a6b8b216e4018b21 Jul 06 19:30:41.656447 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:41.656412 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f8bbd8b45-cl5kr" event={"ID":"67ebdf3b-f451-492f-aaf8-1e668ddfbde6","Type":"ContainerStarted","Data":"f54e380ab25bae84305c80de9daab5cb2a271c0a6dc3884fdb0a4d3761f22517"} Jul 06 19:30:41.656823 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:41.656466 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f8bbd8b45-cl5kr" event={"ID":"67ebdf3b-f451-492f-aaf8-1e668ddfbde6","Type":"ContainerStarted","Data":"ae8219daa3ca57ac4944a40197155172c7f3f886457aea26a6b8b216e4018b21"} Jul 06 19:30:41.676436 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:41.676381 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f8bbd8b45-cl5kr" podStartSLOduration=1.676364858 podStartE2EDuration="1.676364858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:30:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:30:41.674194017 +0000 UTC m=+176.541869574" watchObservedRunningTime="2026-07-06 19:30:41.676364858 +0000 UTC m=+176.544040414" Jul 06 19:30:42.300009 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:42.299948 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:30:50.760028 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:50.759990 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:50.760567 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:50.760090 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:50.764885 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:50.764856 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:51.725002 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:51.724973 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:30:51.776315 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:30:51.776266 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-744b7f446c-srf8s"] Jul 06 19:31:16.799693 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:16.799631 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-744b7f446c-srf8s" podUID="0eb7f2fd-8918-419b-b2a5-e2386e4a948e" containerName="console" containerID="cri-o://ecd1f79e008bd2f73ce1cca7357c4817f6979a2be649fff2a7cee84ee40b5d3a" gracePeriod=15 Jul 06 19:31:17.040633 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.040605 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:31:17.174553 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.174524 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-serving-cert\") pod \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " Jul 06 19:31:17.174714 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.174590 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-trusted-ca-bundle\") pod \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " Jul 06 19:31:17.174714 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.174623 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-oauth-config\") pod \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " Jul 06 19:31:17.174714 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.174677 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-config\") pod \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " Jul 06 19:31:17.174714 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.174707 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtg84\" (UniqueName: \"kubernetes.io/projected/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-kube-api-access-mtg84\") pod \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " Jul 06 19:31:17.174864 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.174729 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-oauth-serving-cert\") pod \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " Jul 06 19:31:17.174864 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.174744 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-service-ca\") pod \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\" (UID: \"0eb7f2fd-8918-419b-b2a5-e2386e4a948e\") " Jul 06 19:31:17.175195 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.175078 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-service-ca" (OuterVolumeSpecName: "service-ca") pod "0eb7f2fd-8918-419b-b2a5-e2386e4a948e" (UID: "0eb7f2fd-8918-419b-b2a5-e2386e4a948e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:31:17.175377 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.175103 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0eb7f2fd-8918-419b-b2a5-e2386e4a948e" (UID: "0eb7f2fd-8918-419b-b2a5-e2386e4a948e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:31:17.175377 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.175275 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0eb7f2fd-8918-419b-b2a5-e2386e4a948e" (UID: "0eb7f2fd-8918-419b-b2a5-e2386e4a948e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:31:17.175377 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.175279 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-config" (OuterVolumeSpecName: "console-config") pod "0eb7f2fd-8918-419b-b2a5-e2386e4a948e" (UID: "0eb7f2fd-8918-419b-b2a5-e2386e4a948e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:31:17.177010 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.176980 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0eb7f2fd-8918-419b-b2a5-e2386e4a948e" (UID: "0eb7f2fd-8918-419b-b2a5-e2386e4a948e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:31:17.177010 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.176988 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-kube-api-access-mtg84" (OuterVolumeSpecName: "kube-api-access-mtg84") pod "0eb7f2fd-8918-419b-b2a5-e2386e4a948e" (UID: "0eb7f2fd-8918-419b-b2a5-e2386e4a948e"). InnerVolumeSpecName "kube-api-access-mtg84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:31:17.177140 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.177041 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0eb7f2fd-8918-419b-b2a5-e2386e4a948e" (UID: "0eb7f2fd-8918-419b-b2a5-e2386e4a948e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:31:17.276256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.276220 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtg84\" (UniqueName: \"kubernetes.io/projected/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-kube-api-access-mtg84\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:31:17.276256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.276251 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-oauth-serving-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:31:17.276256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.276263 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-service-ca\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:31:17.276516 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.276271 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-serving-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:31:17.276516 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.276282 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-trusted-ca-bundle\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:31:17.276516 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.276316 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-oauth-config\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:31:17.276516 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.276326 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0eb7f2fd-8918-419b-b2a5-e2386e4a948e-console-config\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:31:17.830486 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.830454 2561 generic.go:356] "Generic (PLEG): container finished" podID="0eb7f2fd-8918-419b-b2a5-e2386e4a948e" containerID="ecd1f79e008bd2f73ce1cca7357c4817f6979a2be649fff2a7cee84ee40b5d3a" exitCode=0 Jul 06 19:31:17.830895 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.830499 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-744b7f446c-srf8s" Jul 06 19:31:17.830895 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.830514 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744b7f446c-srf8s" event={"ID":"0eb7f2fd-8918-419b-b2a5-e2386e4a948e","Type":"ContainerDied","Data":"ecd1f79e008bd2f73ce1cca7357c4817f6979a2be649fff2a7cee84ee40b5d3a"} Jul 06 19:31:17.830895 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.830561 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-744b7f446c-srf8s" event={"ID":"0eb7f2fd-8918-419b-b2a5-e2386e4a948e","Type":"ContainerDied","Data":"76415643e94eeeada8c62026530bdca6adba06d29638fa3c0d0caeeb44734f0c"} Jul 06 19:31:17.830895 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.830584 2561 scope.go:117] "RemoveContainer" containerID="ecd1f79e008bd2f73ce1cca7357c4817f6979a2be649fff2a7cee84ee40b5d3a" Jul 06 19:31:17.838897 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.838876 2561 scope.go:117] "RemoveContainer" containerID="ecd1f79e008bd2f73ce1cca7357c4817f6979a2be649fff2a7cee84ee40b5d3a" Jul 06 19:31:17.839186 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:31:17.839165 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecd1f79e008bd2f73ce1cca7357c4817f6979a2be649fff2a7cee84ee40b5d3a\": container with ID starting with ecd1f79e008bd2f73ce1cca7357c4817f6979a2be649fff2a7cee84ee40b5d3a not found: ID does not exist" containerID="ecd1f79e008bd2f73ce1cca7357c4817f6979a2be649fff2a7cee84ee40b5d3a" Jul 06 19:31:17.839262 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.839194 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecd1f79e008bd2f73ce1cca7357c4817f6979a2be649fff2a7cee84ee40b5d3a"} err="failed to get container status \"ecd1f79e008bd2f73ce1cca7357c4817f6979a2be649fff2a7cee84ee40b5d3a\": rpc error: code = NotFound desc = could not find container \"ecd1f79e008bd2f73ce1cca7357c4817f6979a2be649fff2a7cee84ee40b5d3a\": container with ID starting with ecd1f79e008bd2f73ce1cca7357c4817f6979a2be649fff2a7cee84ee40b5d3a not found: ID does not exist" Jul 06 19:31:17.857179 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.857147 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-744b7f446c-srf8s"] Jul 06 19:31:17.863014 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:17.862986 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-744b7f446c-srf8s"] Jul 06 19:31:19.685770 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:19.685739 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0eb7f2fd-8918-419b-b2a5-e2386e4a948e" path="/var/lib/kubelet/pods/0eb7f2fd-8918-419b-b2a5-e2386e4a948e/volumes" Jul 06 19:31:37.300278 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:37.300240 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:31:37.316501 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:37.316476 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:31:37.925678 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:37.925653 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:31:47.502763 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.502730 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57746b86db-whs5h"] Jul 06 19:31:47.503168 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.503153 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0eb7f2fd-8918-419b-b2a5-e2386e4a948e" containerName="console" Jul 06 19:31:47.503243 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.503170 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0eb7f2fd-8918-419b-b2a5-e2386e4a948e" containerName="console" Jul 06 19:31:47.503243 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.503230 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0eb7f2fd-8918-419b-b2a5-e2386e4a948e" containerName="console" Jul 06 19:31:47.507422 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.507407 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.524303 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.524271 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57746b86db-whs5h"] Jul 06 19:31:47.630554 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.630517 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-console-config\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.630554 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.630553 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-trusted-ca-bundle\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.630832 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.630578 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjfq4\" (UniqueName: \"kubernetes.io/projected/082cf40c-a116-4e8d-a556-8e04a6205e49-kube-api-access-zjfq4\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.630832 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.630651 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/082cf40c-a116-4e8d-a556-8e04a6205e49-console-oauth-config\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.630832 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.630674 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-oauth-serving-cert\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.630832 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.630691 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/082cf40c-a116-4e8d-a556-8e04a6205e49-console-serving-cert\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.630832 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.630712 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-service-ca\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.731659 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.731635 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-trusted-ca-bundle\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.731810 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.731671 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjfq4\" (UniqueName: \"kubernetes.io/projected/082cf40c-a116-4e8d-a556-8e04a6205e49-kube-api-access-zjfq4\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.731810 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.731707 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/082cf40c-a116-4e8d-a556-8e04a6205e49-console-oauth-config\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.731893 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.731815 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-oauth-serving-cert\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.731893 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.731857 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/082cf40c-a116-4e8d-a556-8e04a6205e49-console-serving-cert\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.731990 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.731894 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-service-ca\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.731990 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.731975 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-console-config\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.732590 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.732562 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-oauth-serving-cert\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.732724 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.732568 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-service-ca\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.732724 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.732601 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-trusted-ca-bundle\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.732724 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.732682 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-console-config\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.734155 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.734124 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/082cf40c-a116-4e8d-a556-8e04a6205e49-console-oauth-config\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.734272 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.734261 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/082cf40c-a116-4e8d-a556-8e04a6205e49-console-serving-cert\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.740284 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.740265 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjfq4\" (UniqueName: \"kubernetes.io/projected/082cf40c-a116-4e8d-a556-8e04a6205e49-kube-api-access-zjfq4\") pod \"console-57746b86db-whs5h\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.816840 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.816769 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:47.945981 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:47.945955 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57746b86db-whs5h"] Jul 06 19:31:47.948070 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:31:47.948042 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod082cf40c_a116_4e8d_a556_8e04a6205e49.slice/crio-718642e99a5c5a8a37c894d3dfdbea1f114c7f93d920c4095ea30fd5fd73d040 WatchSource:0}: Error finding container 718642e99a5c5a8a37c894d3dfdbea1f114c7f93d920c4095ea30fd5fd73d040: Status 404 returned error can't find the container with id 718642e99a5c5a8a37c894d3dfdbea1f114c7f93d920c4095ea30fd5fd73d040 Jul 06 19:31:48.955040 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:48.955007 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57746b86db-whs5h" event={"ID":"082cf40c-a116-4e8d-a556-8e04a6205e49","Type":"ContainerStarted","Data":"c8aef8bd0d46198777d3dbd8149e121d4262300b27517c44d6dc6be8079b9083"} Jul 06 19:31:48.955450 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:48.955056 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57746b86db-whs5h" event={"ID":"082cf40c-a116-4e8d-a556-8e04a6205e49","Type":"ContainerStarted","Data":"718642e99a5c5a8a37c894d3dfdbea1f114c7f93d920c4095ea30fd5fd73d040"} Jul 06 19:31:48.975547 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:48.975498 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57746b86db-whs5h" podStartSLOduration=1.9754833710000002 podStartE2EDuration="1.975483371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:31:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:31:48.973930798 +0000 UTC m=+243.841606355" watchObservedRunningTime="2026-07-06 19:31:48.975483371 +0000 UTC m=+243.843158927" Jul 06 19:31:57.817109 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:57.817073 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:57.817511 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:57.817238 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:57.821779 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:57.821760 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:57.993398 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:57.993373 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:31:58.052263 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:31:58.052233 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f8bbd8b45-cl5kr"] Jul 06 19:32:23.080558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.080452 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7f8bbd8b45-cl5kr" podUID="67ebdf3b-f451-492f-aaf8-1e668ddfbde6" containerName="console" containerID="cri-o://f54e380ab25bae84305c80de9daab5cb2a271c0a6dc3884fdb0a4d3761f22517" gracePeriod=15 Jul 06 19:32:23.323387 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.323362 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:32:23.334376 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.334317 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-oauth-config\") pod \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " Jul 06 19:32:23.334376 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.334367 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-config\") pod \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " Jul 06 19:32:23.334779 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.334393 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-service-ca\") pod \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " Jul 06 19:32:23.334779 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.334433 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-trusted-ca-bundle\") pod \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " Jul 06 19:32:23.334779 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.334479 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-serving-cert\") pod \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " Jul 06 19:32:23.334779 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.334510 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6zzv\" (UniqueName: \"kubernetes.io/projected/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-kube-api-access-z6zzv\") pod \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " Jul 06 19:32:23.334779 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.334565 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-oauth-serving-cert\") pod \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\" (UID: \"67ebdf3b-f451-492f-aaf8-1e668ddfbde6\") " Jul 06 19:32:23.334779 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.334734 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-service-ca" (OuterVolumeSpecName: "service-ca") pod "67ebdf3b-f451-492f-aaf8-1e668ddfbde6" (UID: "67ebdf3b-f451-492f-aaf8-1e668ddfbde6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:32:23.335090 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.334836 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "67ebdf3b-f451-492f-aaf8-1e668ddfbde6" (UID: "67ebdf3b-f451-492f-aaf8-1e668ddfbde6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:32:23.335090 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.334866 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-service-ca\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:32:23.335090 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.335080 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "67ebdf3b-f451-492f-aaf8-1e668ddfbde6" (UID: "67ebdf3b-f451-492f-aaf8-1e668ddfbde6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:32:23.335214 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.335088 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-config" (OuterVolumeSpecName: "console-config") pod "67ebdf3b-f451-492f-aaf8-1e668ddfbde6" (UID: "67ebdf3b-f451-492f-aaf8-1e668ddfbde6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:32:23.336625 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.336596 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "67ebdf3b-f451-492f-aaf8-1e668ddfbde6" (UID: "67ebdf3b-f451-492f-aaf8-1e668ddfbde6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:32:23.336882 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.336855 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-kube-api-access-z6zzv" (OuterVolumeSpecName: "kube-api-access-z6zzv") pod "67ebdf3b-f451-492f-aaf8-1e668ddfbde6" (UID: "67ebdf3b-f451-492f-aaf8-1e668ddfbde6"). InnerVolumeSpecName "kube-api-access-z6zzv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:32:23.337002 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.336965 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "67ebdf3b-f451-492f-aaf8-1e668ddfbde6" (UID: "67ebdf3b-f451-492f-aaf8-1e668ddfbde6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:32:23.435580 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.435541 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-serving-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:32:23.435580 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.435574 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6zzv\" (UniqueName: \"kubernetes.io/projected/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-kube-api-access-z6zzv\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:32:23.435580 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.435587 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-oauth-serving-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:32:23.435818 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.435599 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-oauth-config\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:32:23.435818 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.435611 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-console-config\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:32:23.435818 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:23.435624 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ebdf3b-f451-492f-aaf8-1e668ddfbde6-trusted-ca-bundle\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:32:24.093406 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:24.093284 2561 generic.go:356] "Generic (PLEG): container finished" podID="67ebdf3b-f451-492f-aaf8-1e668ddfbde6" containerID="f54e380ab25bae84305c80de9daab5cb2a271c0a6dc3884fdb0a4d3761f22517" exitCode=0 Jul 06 19:32:24.093406 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:24.093343 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f8bbd8b45-cl5kr" Jul 06 19:32:24.093406 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:24.093370 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f8bbd8b45-cl5kr" event={"ID":"67ebdf3b-f451-492f-aaf8-1e668ddfbde6","Type":"ContainerDied","Data":"f54e380ab25bae84305c80de9daab5cb2a271c0a6dc3884fdb0a4d3761f22517"} Jul 06 19:32:24.093406 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:24.093411 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f8bbd8b45-cl5kr" event={"ID":"67ebdf3b-f451-492f-aaf8-1e668ddfbde6","Type":"ContainerDied","Data":"ae8219daa3ca57ac4944a40197155172c7f3f886457aea26a6b8b216e4018b21"} Jul 06 19:32:24.093972 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:24.093430 2561 scope.go:117] "RemoveContainer" containerID="f54e380ab25bae84305c80de9daab5cb2a271c0a6dc3884fdb0a4d3761f22517" Jul 06 19:32:24.101567 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:24.101546 2561 scope.go:117] "RemoveContainer" containerID="f54e380ab25bae84305c80de9daab5cb2a271c0a6dc3884fdb0a4d3761f22517" Jul 06 19:32:24.101824 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:32:24.101806 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f54e380ab25bae84305c80de9daab5cb2a271c0a6dc3884fdb0a4d3761f22517\": container with ID starting with f54e380ab25bae84305c80de9daab5cb2a271c0a6dc3884fdb0a4d3761f22517 not found: ID does not exist" containerID="f54e380ab25bae84305c80de9daab5cb2a271c0a6dc3884fdb0a4d3761f22517" Jul 06 19:32:24.101903 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:24.101830 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f54e380ab25bae84305c80de9daab5cb2a271c0a6dc3884fdb0a4d3761f22517"} err="failed to get container status \"f54e380ab25bae84305c80de9daab5cb2a271c0a6dc3884fdb0a4d3761f22517\": rpc error: code = NotFound desc = could not find container \"f54e380ab25bae84305c80de9daab5cb2a271c0a6dc3884fdb0a4d3761f22517\": container with ID starting with f54e380ab25bae84305c80de9daab5cb2a271c0a6dc3884fdb0a4d3761f22517 not found: ID does not exist" Jul 06 19:32:24.111147 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:24.111117 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f8bbd8b45-cl5kr"] Jul 06 19:32:24.115515 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:24.115483 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f8bbd8b45-cl5kr"] Jul 06 19:32:25.685210 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:25.685179 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67ebdf3b-f451-492f-aaf8-1e668ddfbde6" path="/var/lib/kubelet/pods/67ebdf3b-f451-492f-aaf8-1e668ddfbde6/volumes" Jul 06 19:32:45.526300 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:32:45.526270 2561 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 19:33:23.599444 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.599406 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dp5pc"] Jul 06 19:33:23.601998 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.599868 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67ebdf3b-f451-492f-aaf8-1e668ddfbde6" containerName="console" Jul 06 19:33:23.601998 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.599884 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="67ebdf3b-f451-492f-aaf8-1e668ddfbde6" containerName="console" Jul 06 19:33:23.601998 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.599941 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="67ebdf3b-f451-492f-aaf8-1e668ddfbde6" containerName="console" Jul 06 19:33:23.602860 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.602845 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dp5pc" Jul 06 19:33:23.605611 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.605588 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 19:33:23.606172 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.606151 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 19:33:23.606270 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.606151 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-jmbs8\"" Jul 06 19:33:23.614727 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.614702 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dp5pc"] Jul 06 19:33:23.637434 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.637403 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f99e02d-5538-4419-8c58-8eaeca14e58e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dp5pc\" (UID: \"2f99e02d-5538-4419-8c58-8eaeca14e58e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dp5pc" Jul 06 19:33:23.637579 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.637457 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6hzj\" (UniqueName: \"kubernetes.io/projected/2f99e02d-5538-4419-8c58-8eaeca14e58e-kube-api-access-w6hzj\") pod \"cert-manager-webhook-6486ffd997-dp5pc\" (UID: \"2f99e02d-5538-4419-8c58-8eaeca14e58e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dp5pc" Jul 06 19:33:23.738586 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.738548 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f99e02d-5538-4419-8c58-8eaeca14e58e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dp5pc\" (UID: \"2f99e02d-5538-4419-8c58-8eaeca14e58e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dp5pc" Jul 06 19:33:23.738796 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.738601 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6hzj\" (UniqueName: \"kubernetes.io/projected/2f99e02d-5538-4419-8c58-8eaeca14e58e-kube-api-access-w6hzj\") pod \"cert-manager-webhook-6486ffd997-dp5pc\" (UID: \"2f99e02d-5538-4419-8c58-8eaeca14e58e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dp5pc" Jul 06 19:33:23.748590 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.748558 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f99e02d-5538-4419-8c58-8eaeca14e58e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dp5pc\" (UID: \"2f99e02d-5538-4419-8c58-8eaeca14e58e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dp5pc" Jul 06 19:33:23.748729 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.748676 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6hzj\" (UniqueName: \"kubernetes.io/projected/2f99e02d-5538-4419-8c58-8eaeca14e58e-kube-api-access-w6hzj\") pod \"cert-manager-webhook-6486ffd997-dp5pc\" (UID: \"2f99e02d-5538-4419-8c58-8eaeca14e58e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dp5pc" Jul 06 19:33:23.924871 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:23.924835 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dp5pc" Jul 06 19:33:24.062903 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:24.060474 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dp5pc"] Jul 06 19:33:24.064403 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:33:24.064368 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f99e02d_5538_4419_8c58_8eaeca14e58e.slice/crio-e3701817cb1d238bdc4db654bc891c3a2839bf5753bf2bd3ec46a511d8800781 WatchSource:0}: Error finding container e3701817cb1d238bdc4db654bc891c3a2839bf5753bf2bd3ec46a511d8800781: Status 404 returned error can't find the container with id e3701817cb1d238bdc4db654bc891c3a2839bf5753bf2bd3ec46a511d8800781 Jul 06 19:33:24.066518 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:24.066496 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:33:24.337692 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:24.337607 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dp5pc" event={"ID":"2f99e02d-5538-4419-8c58-8eaeca14e58e","Type":"ContainerStarted","Data":"e3701817cb1d238bdc4db654bc891c3a2839bf5753bf2bd3ec46a511d8800781"} Jul 06 19:33:27.353849 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:27.353758 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dp5pc" event={"ID":"2f99e02d-5538-4419-8c58-8eaeca14e58e","Type":"ContainerStarted","Data":"d446bd55808d25da6a1f9c606056a033b831a321067772b44fed837e4471c553"} Jul 06 19:33:27.353849 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:27.353822 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dp5pc" Jul 06 19:33:27.370065 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:27.369997 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-dp5pc" podStartSLOduration=1.482834429 podStartE2EDuration="4.369977443s" totalImagesPullingTime="2.887143014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:33:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:33:27.369230001 +0000 UTC m=+342.236905558" watchObservedRunningTime="2026-07-06 19:33:27.369977443 +0000 UTC m=+342.237653000" Jul 06 19:33:33.361043 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:33.361008 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dp5pc" Jul 06 19:33:36.386525 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:36.386493 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-krnrw"] Jul 06 19:33:36.389818 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:36.389794 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-krnrw" Jul 06 19:33:36.392464 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:36.392443 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 19:33:36.392464 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:36.392456 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:33:36.392608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:36.392443 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-rc7j4\"" Jul 06 19:33:36.399761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:36.399734 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-krnrw"] Jul 06 19:33:36.458696 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:36.458662 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ttdm\" (UniqueName: \"kubernetes.io/projected/3447ee19-b873-4c82-8293-38d3be493a0f-kube-api-access-9ttdm\") pod \"openshift-lws-operator-fd8ccff4c-krnrw\" (UID: \"3447ee19-b873-4c82-8293-38d3be493a0f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-krnrw" Jul 06 19:33:36.458869 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:36.458737 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3447ee19-b873-4c82-8293-38d3be493a0f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-krnrw\" (UID: \"3447ee19-b873-4c82-8293-38d3be493a0f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-krnrw" Jul 06 19:33:36.560061 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:36.560024 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3447ee19-b873-4c82-8293-38d3be493a0f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-krnrw\" (UID: \"3447ee19-b873-4c82-8293-38d3be493a0f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-krnrw" Jul 06 19:33:36.560233 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:36.560100 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ttdm\" (UniqueName: \"kubernetes.io/projected/3447ee19-b873-4c82-8293-38d3be493a0f-kube-api-access-9ttdm\") pod \"openshift-lws-operator-fd8ccff4c-krnrw\" (UID: \"3447ee19-b873-4c82-8293-38d3be493a0f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-krnrw" Jul 06 19:33:36.560464 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:36.560441 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3447ee19-b873-4c82-8293-38d3be493a0f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-krnrw\" (UID: \"3447ee19-b873-4c82-8293-38d3be493a0f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-krnrw" Jul 06 19:33:36.570349 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:36.570315 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ttdm\" (UniqueName: \"kubernetes.io/projected/3447ee19-b873-4c82-8293-38d3be493a0f-kube-api-access-9ttdm\") pod \"openshift-lws-operator-fd8ccff4c-krnrw\" (UID: \"3447ee19-b873-4c82-8293-38d3be493a0f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-krnrw" Jul 06 19:33:36.699031 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:36.698998 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-krnrw" Jul 06 19:33:36.836549 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:36.836522 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-krnrw"] Jul 06 19:33:36.839057 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:33:36.839025 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3447ee19_b873_4c82_8293_38d3be493a0f.slice/crio-6bac41c3111ef7fe6912bb90bcfc9f13ae33220404f9fc500733dc291a52aca9 WatchSource:0}: Error finding container 6bac41c3111ef7fe6912bb90bcfc9f13ae33220404f9fc500733dc291a52aca9: Status 404 returned error can't find the container with id 6bac41c3111ef7fe6912bb90bcfc9f13ae33220404f9fc500733dc291a52aca9 Jul 06 19:33:37.396515 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:37.396473 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-krnrw" event={"ID":"3447ee19-b873-4c82-8293-38d3be493a0f","Type":"ContainerStarted","Data":"6bac41c3111ef7fe6912bb90bcfc9f13ae33220404f9fc500733dc291a52aca9"} Jul 06 19:33:40.411762 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:40.411722 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-krnrw" event={"ID":"3447ee19-b873-4c82-8293-38d3be493a0f","Type":"ContainerStarted","Data":"14ae3234ed15e781e72a190351c2e16ffae44c525a6e5cf2a32d4ee9f736c8ba"} Jul 06 19:33:40.430143 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:40.430090 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-krnrw" podStartSLOduration=1.012638713 podStartE2EDuration="4.430072302s" totalImagesPullingTime="3.417433589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:33:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:33:40.428434889 +0000 UTC m=+355.296110446" watchObservedRunningTime="2026-07-06 19:33:40.430072302 +0000 UTC m=+355.297747855" Jul 06 19:33:56.003769 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.003731 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz"] Jul 06 19:33:56.007380 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.007359 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.012700 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.012546 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 19:33:56.012700 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.012572 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 19:33:56.012916 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.012819 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 19:33:56.012916 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.012904 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 19:33:56.013023 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.012971 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-csv8h\"" Jul 06 19:33:56.028867 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.028840 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz"] Jul 06 19:33:56.139689 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.139645 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945a5232-1920-431c-a151-e36544a90d6f-webhook-cert\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.139689 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.139688 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkz7p\" (UniqueName: \"kubernetes.io/projected/945a5232-1920-431c-a151-e36544a90d6f-kube-api-access-nkz7p\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.139942 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.139764 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/945a5232-1920-431c-a151-e36544a90d6f-manifests\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.139942 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.139824 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/945a5232-1920-431c-a151-e36544a90d6f-tmp\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.139942 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.139848 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/945a5232-1920-431c-a151-e36544a90d6f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.240754 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.240713 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/945a5232-1920-431c-a151-e36544a90d6f-manifests\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.240955 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.240765 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/945a5232-1920-431c-a151-e36544a90d6f-tmp\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.240955 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.240901 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/945a5232-1920-431c-a151-e36544a90d6f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.241076 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.241044 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945a5232-1920-431c-a151-e36544a90d6f-webhook-cert\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.241137 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.241077 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkz7p\" (UniqueName: \"kubernetes.io/projected/945a5232-1920-431c-a151-e36544a90d6f-kube-api-access-nkz7p\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.241137 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.241097 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/945a5232-1920-431c-a151-e36544a90d6f-manifests\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.243187 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.243159 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/945a5232-1920-431c-a151-e36544a90d6f-tmp\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.243324 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.243234 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/945a5232-1920-431c-a151-e36544a90d6f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.243367 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.243335 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945a5232-1920-431c-a151-e36544a90d6f-webhook-cert\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.255230 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.255122 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkz7p\" (UniqueName: \"kubernetes.io/projected/945a5232-1920-431c-a151-e36544a90d6f-kube-api-access-nkz7p\") pod \"opendatahub-operator-controller-manager-57b585888c-cwgsz\" (UID: \"945a5232-1920-431c-a151-e36544a90d6f\") " pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.317637 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.317599 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:33:56.463019 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.462994 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz"] Jul 06 19:33:56.465758 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:33:56.465731 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945a5232_1920_431c_a151_e36544a90d6f.slice/crio-91bdb4ea5885bed67e481a94d98092bf0df22524f6c6424f395b5770f6833bac WatchSource:0}: Error finding container 91bdb4ea5885bed67e481a94d98092bf0df22524f6c6424f395b5770f6833bac: Status 404 returned error can't find the container with id 91bdb4ea5885bed67e481a94d98092bf0df22524f6c6424f395b5770f6833bac Jul 06 19:33:56.478871 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:56.478839 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" event={"ID":"945a5232-1920-431c-a151-e36544a90d6f","Type":"ContainerStarted","Data":"91bdb4ea5885bed67e481a94d98092bf0df22524f6c6424f395b5770f6833bac"} Jul 06 19:33:59.495760 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:33:59.495725 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" event={"ID":"945a5232-1920-431c-a151-e36544a90d6f","Type":"ContainerStarted","Data":"9f5cf4f4f3f081f2b98a921a37c5265d51d1ae61dfbefa0bf7d7b472877f4c71"} Jul 06 19:34:00.501436 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:00.501398 2561 generic.go:356] "Generic (PLEG): container finished" podID="945a5232-1920-431c-a151-e36544a90d6f" containerID="9f5cf4f4f3f081f2b98a921a37c5265d51d1ae61dfbefa0bf7d7b472877f4c71" exitCode=0 Jul 06 19:34:00.501790 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:00.501462 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" event={"ID":"945a5232-1920-431c-a151-e36544a90d6f","Type":"ContainerDied","Data":"9f5cf4f4f3f081f2b98a921a37c5265d51d1ae61dfbefa0bf7d7b472877f4c71"} Jul 06 19:34:01.507874 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:01.507832 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" event={"ID":"945a5232-1920-431c-a151-e36544a90d6f","Type":"ContainerStarted","Data":"7d15a42ea56a7dbc2c7fcdd1e1f99974fa11b7be8a4dcbd8e9cb20ef1264197b"} Jul 06 19:34:01.508444 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:01.507935 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:34:01.532149 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:01.532090 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" podStartSLOduration=2.034568356 podStartE2EDuration="6.532065763s" totalImagesPullingTime="3.490811695s" totalInitContainerRuntime="1.006685712s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:33:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:01.53035133 +0000 UTC m=+376.398026883" watchObservedRunningTime="2026-07-06 19:34:01.532065763 +0000 UTC m=+376.399741320" Jul 06 19:34:12.514668 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:12.514629 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-57b585888c-cwgsz" Jul 06 19:34:17.237176 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.237138 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz"] Jul 06 19:34:17.243102 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.243074 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:17.249726 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.249504 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 19:34:17.249726 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.249551 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 19:34:17.250423 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.250404 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 19:34:17.251398 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.251379 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-zfg8p\"" Jul 06 19:34:17.259926 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.259901 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz"] Jul 06 19:34:17.321091 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.321053 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5-cert\") pod \"lws-controller-manager-7b7d6b4d5f-gldgz\" (UID: \"2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5\") " pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:17.321259 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.321129 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql2m6\" (UniqueName: \"kubernetes.io/projected/2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5-kube-api-access-ql2m6\") pod \"lws-controller-manager-7b7d6b4d5f-gldgz\" (UID: \"2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5\") " pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:17.321259 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.321173 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5-manager-config\") pod \"lws-controller-manager-7b7d6b4d5f-gldgz\" (UID: \"2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5\") " pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:17.321259 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.321241 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5-metrics-cert\") pod \"lws-controller-manager-7b7d6b4d5f-gldgz\" (UID: \"2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5\") " pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:17.422559 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.422512 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5-cert\") pod \"lws-controller-manager-7b7d6b4d5f-gldgz\" (UID: \"2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5\") " pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:17.422763 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.422584 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql2m6\" (UniqueName: \"kubernetes.io/projected/2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5-kube-api-access-ql2m6\") pod \"lws-controller-manager-7b7d6b4d5f-gldgz\" (UID: \"2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5\") " pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:17.422763 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.422618 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5-manager-config\") pod \"lws-controller-manager-7b7d6b4d5f-gldgz\" (UID: \"2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5\") " pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:17.422763 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.422652 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5-metrics-cert\") pod \"lws-controller-manager-7b7d6b4d5f-gldgz\" (UID: \"2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5\") " pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:17.423369 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.423345 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5-manager-config\") pod \"lws-controller-manager-7b7d6b4d5f-gldgz\" (UID: \"2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5\") " pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:17.424999 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.424972 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5-cert\") pod \"lws-controller-manager-7b7d6b4d5f-gldgz\" (UID: \"2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5\") " pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:17.425127 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.425111 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5-metrics-cert\") pod \"lws-controller-manager-7b7d6b4d5f-gldgz\" (UID: \"2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5\") " pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:17.443867 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.443838 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql2m6\" (UniqueName: \"kubernetes.io/projected/2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5-kube-api-access-ql2m6\") pod \"lws-controller-manager-7b7d6b4d5f-gldgz\" (UID: \"2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5\") " pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:17.553262 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.553164 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:17.729854 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:17.729827 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz"] Jul 06 19:34:17.730949 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:34:17.730910 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2433e1cb_2cc2_4ba5_b04e_0f78b85d55e5.slice/crio-e30fbfeb25c16a6e7d6d6c1f76bb8032111d684a01b209ca4c7058a6de813d3b WatchSource:0}: Error finding container e30fbfeb25c16a6e7d6d6c1f76bb8032111d684a01b209ca4c7058a6de813d3b: Status 404 returned error can't find the container with id e30fbfeb25c16a6e7d6d6c1f76bb8032111d684a01b209ca4c7058a6de813d3b Jul 06 19:34:18.591727 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:18.591690 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" event={"ID":"2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5","Type":"ContainerStarted","Data":"e30fbfeb25c16a6e7d6d6c1f76bb8032111d684a01b209ca4c7058a6de813d3b"} Jul 06 19:34:20.602734 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:20.602695 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" event={"ID":"2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5","Type":"ContainerStarted","Data":"195761aa2a2c60e4a74119511dd1bf346c8f0ea814c8db56db06dad8d67d0ce2"} Jul 06 19:34:20.603129 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:20.602817 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:34:20.621523 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:20.621474 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" podStartSLOduration=1.6349686810000001 podStartE2EDuration="3.621457933s" totalImagesPullingTime="1.986489252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:20.619304384 +0000 UTC m=+395.486979932" watchObservedRunningTime="2026-07-06 19:34:20.621457933 +0000 UTC m=+395.489133490" Jul 06 19:34:21.831876 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.831847 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6"] Jul 06 19:34:21.835385 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.835363 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.837921 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.837904 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 19:34:21.838081 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.837933 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 19:34:21.838081 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.837907 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 19:34:21.838179 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.838163 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-s7sdc\"" Jul 06 19:34:21.849893 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.849869 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6"] Jul 06 19:34:21.868505 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.868477 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.868505 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.868515 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.868732 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.868542 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.868732 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.868571 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.868732 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.868594 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnpx6\" (UniqueName: \"kubernetes.io/projected/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-kube-api-access-qnpx6\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.868732 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.868637 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.868732 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.868667 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.868991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.868736 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.868991 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.868791 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.969857 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.969822 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.970025 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.969874 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.970025 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.969912 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.970025 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.969931 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.970025 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.969955 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnpx6\" (UniqueName: \"kubernetes.io/projected/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-kube-api-access-qnpx6\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.970025 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.969978 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.970025 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.970021 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.970399 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.970176 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.970399 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.970244 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.970399 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.970277 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.970399 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.970360 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.970631 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.970614 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.970686 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.970642 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.970874 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.970850 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.972340 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.972314 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.972543 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.972527 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.979157 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.979129 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnpx6\" (UniqueName: \"kubernetes.io/projected/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-kube-api-access-qnpx6\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:21.980680 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:21.980659 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c8354b6-b95c-4ac1-bf88-f9eb8029678b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf479jh6\" (UID: \"8c8354b6-b95c-4ac1-bf88-f9eb8029678b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:22.147970 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:22.147879 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:22.283115 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:22.283084 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6"] Jul 06 19:34:22.286279 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:34:22.286249 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c8354b6_b95c_4ac1_bf88_f9eb8029678b.slice/crio-9127b8336a7e2aada1486e5385f33add807a685191f9b09fdc4b8761b55a6b38 WatchSource:0}: Error finding container 9127b8336a7e2aada1486e5385f33add807a685191f9b09fdc4b8761b55a6b38: Status 404 returned error can't find the container with id 9127b8336a7e2aada1486e5385f33add807a685191f9b09fdc4b8761b55a6b38 Jul 06 19:34:22.613454 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:22.613416 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" event={"ID":"8c8354b6-b95c-4ac1-bf88-f9eb8029678b","Type":"ContainerStarted","Data":"9127b8336a7e2aada1486e5385f33add807a685191f9b09fdc4b8761b55a6b38"} Jul 06 19:34:24.068488 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.068452 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq"] Jul 06 19:34:24.072440 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.072409 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.075086 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.075063 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 19:34:24.075224 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.075187 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-n8d78\"" Jul 06 19:34:24.075399 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.075374 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 19:34:24.075678 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.075660 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 19:34:24.090034 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.090001 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq"] Jul 06 19:34:24.190932 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.190896 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a005f86f-7fab-4720-af71-c49804fee909-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.191125 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.190943 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a005f86f-7fab-4720-af71-c49804fee909-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.191125 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.190967 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a005f86f-7fab-4720-af71-c49804fee909-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.191125 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.191079 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a005f86f-7fab-4720-af71-c49804fee909-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.191125 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.191120 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsbx6\" (UniqueName: \"kubernetes.io/projected/a005f86f-7fab-4720-af71-c49804fee909-kube-api-access-vsbx6\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.191329 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.191176 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a005f86f-7fab-4720-af71-c49804fee909-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.191329 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.191201 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a005f86f-7fab-4720-af71-c49804fee909-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.292367 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.292326 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a005f86f-7fab-4720-af71-c49804fee909-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.292566 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.292385 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsbx6\" (UniqueName: \"kubernetes.io/projected/a005f86f-7fab-4720-af71-c49804fee909-kube-api-access-vsbx6\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.292566 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.292444 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a005f86f-7fab-4720-af71-c49804fee909-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.292566 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.292469 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a005f86f-7fab-4720-af71-c49804fee909-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.292566 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.292538 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a005f86f-7fab-4720-af71-c49804fee909-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.292566 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.292563 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a005f86f-7fab-4720-af71-c49804fee909-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.292835 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.292587 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a005f86f-7fab-4720-af71-c49804fee909-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.293382 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.293329 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a005f86f-7fab-4720-af71-c49804fee909-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.295006 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.294973 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a005f86f-7fab-4720-af71-c49804fee909-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.295225 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.295206 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a005f86f-7fab-4720-af71-c49804fee909-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.295460 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.295438 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a005f86f-7fab-4720-af71-c49804fee909-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.295549 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.295462 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a005f86f-7fab-4720-af71-c49804fee909-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.316227 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.316191 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a005f86f-7fab-4720-af71-c49804fee909-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.319985 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.319920 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsbx6\" (UniqueName: \"kubernetes.io/projected/a005f86f-7fab-4720-af71-c49804fee909-kube-api-access-vsbx6\") pod \"istiod-openshift-gateway-94bb8fbfd-p8prq\" (UID: \"a005f86f-7fab-4720-af71-c49804fee909\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:24.384359 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:24.384320 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:25.206506 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:25.206458 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:34:25.206844 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:25.206562 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:34:25.206844 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:25.206605 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:34:25.293513 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:25.293480 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq"] Jul 06 19:34:25.296216 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:34:25.296188 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda005f86f_7fab_4720_af71_c49804fee909.slice/crio-c4a54edc5607c0d7bfe2281ebfffbac4b4e362cb7191e9e5ef51a6762e6404d2 WatchSource:0}: Error finding container c4a54edc5607c0d7bfe2281ebfffbac4b4e362cb7191e9e5ef51a6762e6404d2: Status 404 returned error can't find the container with id c4a54edc5607c0d7bfe2281ebfffbac4b4e362cb7191e9e5ef51a6762e6404d2 Jul 06 19:34:25.630724 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:25.630692 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" event={"ID":"a005f86f-7fab-4720-af71-c49804fee909","Type":"ContainerStarted","Data":"c4a54edc5607c0d7bfe2281ebfffbac4b4e362cb7191e9e5ef51a6762e6404d2"} Jul 06 19:34:25.631964 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:25.631930 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" event={"ID":"8c8354b6-b95c-4ac1-bf88-f9eb8029678b","Type":"ContainerStarted","Data":"04133031c82f71be1006f70a67fbaca8bd67ad04657993c84e1044a0e509c811"} Jul 06 19:34:25.694164 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:25.694025 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" podStartSLOduration=1.7760620029999998 podStartE2EDuration="4.694005985s" totalImagesPullingTime="2.917943982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:25.692403987 +0000 UTC m=+400.560079546" watchObservedRunningTime="2026-07-06 19:34:25.694005985 +0000 UTC m=+400.561681542" Jul 06 19:34:26.148719 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:26.148682 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:26.153732 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:26.153707 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:26.639705 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:26.639678 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:26.646832 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:26.646793 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf479jh6" Jul 06 19:34:27.600519 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:27.600483 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:34:27.600687 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:27.600552 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:34:28.650354 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:28.650314 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" event={"ID":"a005f86f-7fab-4720-af71-c49804fee909","Type":"ContainerStarted","Data":"6c003c59ab238692da388ab02e622ed3ec6007d9fd26e7e30fbf96419ccbd461"} Jul 06 19:34:28.650817 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:28.650379 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:28.651831 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:28.651804 2561 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-p8prq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 19:34:28.651975 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:28.651856 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" podUID="a005f86f-7fab-4720-af71-c49804fee909" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 19:34:28.673678 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:28.673628 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" podStartSLOduration=2.372055521 podStartE2EDuration="4.673610668s" totalImagesPullingTime="2.301555147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:28.672915748 +0000 UTC m=+403.540591306" watchObservedRunningTime="2026-07-06 19:34:28.673610668 +0000 UTC m=+403.541286226" Jul 06 19:34:29.656085 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:29.656055 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8prq" Jul 06 19:34:31.609444 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:34:31.609416 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b7d6b4d5f-gldgz" Jul 06 19:35:44.611669 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.611637 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d84765bc6-zpmzq"] Jul 06 19:35:44.615079 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.615052 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.630580 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.630555 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d84765bc6-zpmzq"] Jul 06 19:35:44.646797 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.646762 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e662b5d2-0861-450e-a28c-7abb338c1ce7-console-serving-cert\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.646952 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.646825 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e662b5d2-0861-450e-a28c-7abb338c1ce7-trusted-ca-bundle\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.646952 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.646898 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e662b5d2-0861-450e-a28c-7abb338c1ce7-console-config\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.646952 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.646944 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e662b5d2-0861-450e-a28c-7abb338c1ce7-oauth-serving-cert\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.647081 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.647002 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e662b5d2-0861-450e-a28c-7abb338c1ce7-console-oauth-config\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.647081 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.647015 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e662b5d2-0861-450e-a28c-7abb338c1ce7-service-ca\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.647081 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.647034 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v4zt\" (UniqueName: \"kubernetes.io/projected/e662b5d2-0861-450e-a28c-7abb338c1ce7-kube-api-access-7v4zt\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.748105 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.748076 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e662b5d2-0861-450e-a28c-7abb338c1ce7-console-serving-cert\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.748328 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.748119 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e662b5d2-0861-450e-a28c-7abb338c1ce7-trusted-ca-bundle\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.748328 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.748146 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e662b5d2-0861-450e-a28c-7abb338c1ce7-console-config\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.748328 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.748178 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e662b5d2-0861-450e-a28c-7abb338c1ce7-oauth-serving-cert\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.748328 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.748217 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e662b5d2-0861-450e-a28c-7abb338c1ce7-console-oauth-config\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.748328 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.748235 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e662b5d2-0861-450e-a28c-7abb338c1ce7-service-ca\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.748328 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.748256 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4zt\" (UniqueName: \"kubernetes.io/projected/e662b5d2-0861-450e-a28c-7abb338c1ce7-kube-api-access-7v4zt\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.749062 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.749030 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e662b5d2-0861-450e-a28c-7abb338c1ce7-oauth-serving-cert\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.749175 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.749102 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e662b5d2-0861-450e-a28c-7abb338c1ce7-service-ca\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.749175 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.749136 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e662b5d2-0861-450e-a28c-7abb338c1ce7-trusted-ca-bundle\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.749276 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.749196 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e662b5d2-0861-450e-a28c-7abb338c1ce7-console-config\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.750656 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.750633 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e662b5d2-0861-450e-a28c-7abb338c1ce7-console-serving-cert\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.750977 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.750954 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e662b5d2-0861-450e-a28c-7abb338c1ce7-console-oauth-config\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.760398 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.760374 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v4zt\" (UniqueName: \"kubernetes.io/projected/e662b5d2-0861-450e-a28c-7abb338c1ce7-kube-api-access-7v4zt\") pod \"console-6d84765bc6-zpmzq\" (UID: \"e662b5d2-0861-450e-a28c-7abb338c1ce7\") " pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:44.925938 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:44.925907 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:45.062661 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:45.062638 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d84765bc6-zpmzq"] Jul 06 19:35:45.064093 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:35:45.064068 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode662b5d2_0861_450e_a28c_7abb338c1ce7.slice/crio-1b01640185a650c089812e42b6467d683c1e5354dda60d2420c20dadd2354411 WatchSource:0}: Error finding container 1b01640185a650c089812e42b6467d683c1e5354dda60d2420c20dadd2354411: Status 404 returned error can't find the container with id 1b01640185a650c089812e42b6467d683c1e5354dda60d2420c20dadd2354411 Jul 06 19:35:46.007499 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:46.007464 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d84765bc6-zpmzq" event={"ID":"e662b5d2-0861-450e-a28c-7abb338c1ce7","Type":"ContainerStarted","Data":"94dd327faeda49637b3a5c52aa736935dfb50e5d63817add1ffac5c36e9e6393"} Jul 06 19:35:46.007853 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:46.007512 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d84765bc6-zpmzq" event={"ID":"e662b5d2-0861-450e-a28c-7abb338c1ce7","Type":"ContainerStarted","Data":"1b01640185a650c089812e42b6467d683c1e5354dda60d2420c20dadd2354411"} Jul 06 19:35:46.032619 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:46.032567 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d84765bc6-zpmzq" podStartSLOduration=2.032550834 podStartE2EDuration="2.032550834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:35:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:35:46.031099992 +0000 UTC m=+480.898775549" watchObservedRunningTime="2026-07-06 19:35:46.032550834 +0000 UTC m=+480.900226391" Jul 06 19:35:54.926265 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:54.926227 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:54.926752 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:54.926340 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:54.930961 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:54.930938 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:55.050453 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:55.050430 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d84765bc6-zpmzq" Jul 06 19:35:55.107896 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:35:55.107867 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57746b86db-whs5h"] Jul 06 19:36:20.127861 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.127822 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-57746b86db-whs5h" podUID="082cf40c-a116-4e8d-a556-8e04a6205e49" containerName="console" containerID="cri-o://c8aef8bd0d46198777d3dbd8149e121d4262300b27517c44d6dc6be8079b9083" gracePeriod=15 Jul 06 19:36:20.369817 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.369797 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:36:20.472803 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.472776 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/082cf40c-a116-4e8d-a556-8e04a6205e49-console-oauth-config\") pod \"082cf40c-a116-4e8d-a556-8e04a6205e49\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " Jul 06 19:36:20.472968 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.472820 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjfq4\" (UniqueName: \"kubernetes.io/projected/082cf40c-a116-4e8d-a556-8e04a6205e49-kube-api-access-zjfq4\") pod \"082cf40c-a116-4e8d-a556-8e04a6205e49\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " Jul 06 19:36:20.472968 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.472838 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/082cf40c-a116-4e8d-a556-8e04a6205e49-console-serving-cert\") pod \"082cf40c-a116-4e8d-a556-8e04a6205e49\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " Jul 06 19:36:20.472968 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.472880 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-trusted-ca-bundle\") pod \"082cf40c-a116-4e8d-a556-8e04a6205e49\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " Jul 06 19:36:20.472968 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.472905 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-service-ca\") pod \"082cf40c-a116-4e8d-a556-8e04a6205e49\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " Jul 06 19:36:20.473180 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.472978 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-console-config\") pod \"082cf40c-a116-4e8d-a556-8e04a6205e49\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " Jul 06 19:36:20.473180 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.473009 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-oauth-serving-cert\") pod \"082cf40c-a116-4e8d-a556-8e04a6205e49\" (UID: \"082cf40c-a116-4e8d-a556-8e04a6205e49\") " Jul 06 19:36:20.473392 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.473362 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-service-ca" (OuterVolumeSpecName: "service-ca") pod "082cf40c-a116-4e8d-a556-8e04a6205e49" (UID: "082cf40c-a116-4e8d-a556-8e04a6205e49"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:36:20.473392 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.473382 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "082cf40c-a116-4e8d-a556-8e04a6205e49" (UID: "082cf40c-a116-4e8d-a556-8e04a6205e49"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:36:20.473506 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.473459 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "082cf40c-a116-4e8d-a556-8e04a6205e49" (UID: "082cf40c-a116-4e8d-a556-8e04a6205e49"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:36:20.473860 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.473832 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-console-config" (OuterVolumeSpecName: "console-config") pod "082cf40c-a116-4e8d-a556-8e04a6205e49" (UID: "082cf40c-a116-4e8d-a556-8e04a6205e49"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:36:20.474944 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.474920 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/082cf40c-a116-4e8d-a556-8e04a6205e49-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "082cf40c-a116-4e8d-a556-8e04a6205e49" (UID: "082cf40c-a116-4e8d-a556-8e04a6205e49"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:36:20.475027 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.474933 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/082cf40c-a116-4e8d-a556-8e04a6205e49-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "082cf40c-a116-4e8d-a556-8e04a6205e49" (UID: "082cf40c-a116-4e8d-a556-8e04a6205e49"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:36:20.475066 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.475025 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/082cf40c-a116-4e8d-a556-8e04a6205e49-kube-api-access-zjfq4" (OuterVolumeSpecName: "kube-api-access-zjfq4") pod "082cf40c-a116-4e8d-a556-8e04a6205e49" (UID: "082cf40c-a116-4e8d-a556-8e04a6205e49"). InnerVolumeSpecName "kube-api-access-zjfq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:36:20.574570 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.574540 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-trusted-ca-bundle\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:36:20.574570 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.574565 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-service-ca\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:36:20.574570 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.574574 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-console-config\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:36:20.574788 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.574582 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/082cf40c-a116-4e8d-a556-8e04a6205e49-oauth-serving-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:36:20.574788 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.574590 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/082cf40c-a116-4e8d-a556-8e04a6205e49-console-oauth-config\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:36:20.574788 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.574598 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjfq4\" (UniqueName: \"kubernetes.io/projected/082cf40c-a116-4e8d-a556-8e04a6205e49-kube-api-access-zjfq4\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:36:20.574788 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:20.574606 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/082cf40c-a116-4e8d-a556-8e04a6205e49-console-serving-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:36:21.171600 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:21.171568 2561 generic.go:356] "Generic (PLEG): container finished" podID="082cf40c-a116-4e8d-a556-8e04a6205e49" containerID="c8aef8bd0d46198777d3dbd8149e121d4262300b27517c44d6dc6be8079b9083" exitCode=0 Jul 06 19:36:21.171993 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:21.171608 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57746b86db-whs5h" Jul 06 19:36:21.171993 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:21.171648 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57746b86db-whs5h" event={"ID":"082cf40c-a116-4e8d-a556-8e04a6205e49","Type":"ContainerDied","Data":"c8aef8bd0d46198777d3dbd8149e121d4262300b27517c44d6dc6be8079b9083"} Jul 06 19:36:21.171993 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:21.171697 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57746b86db-whs5h" event={"ID":"082cf40c-a116-4e8d-a556-8e04a6205e49","Type":"ContainerDied","Data":"718642e99a5c5a8a37c894d3dfdbea1f114c7f93d920c4095ea30fd5fd73d040"} Jul 06 19:36:21.171993 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:21.171717 2561 scope.go:117] "RemoveContainer" containerID="c8aef8bd0d46198777d3dbd8149e121d4262300b27517c44d6dc6be8079b9083" Jul 06 19:36:21.180570 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:21.180556 2561 scope.go:117] "RemoveContainer" containerID="c8aef8bd0d46198777d3dbd8149e121d4262300b27517c44d6dc6be8079b9083" Jul 06 19:36:21.180820 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:36:21.180798 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8aef8bd0d46198777d3dbd8149e121d4262300b27517c44d6dc6be8079b9083\": container with ID starting with c8aef8bd0d46198777d3dbd8149e121d4262300b27517c44d6dc6be8079b9083 not found: ID does not exist" containerID="c8aef8bd0d46198777d3dbd8149e121d4262300b27517c44d6dc6be8079b9083" Jul 06 19:36:21.180908 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:21.180826 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8aef8bd0d46198777d3dbd8149e121d4262300b27517c44d6dc6be8079b9083"} err="failed to get container status \"c8aef8bd0d46198777d3dbd8149e121d4262300b27517c44d6dc6be8079b9083\": rpc error: code = NotFound desc = could not find container \"c8aef8bd0d46198777d3dbd8149e121d4262300b27517c44d6dc6be8079b9083\": container with ID starting with c8aef8bd0d46198777d3dbd8149e121d4262300b27517c44d6dc6be8079b9083 not found: ID does not exist" Jul 06 19:36:21.193761 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:21.193742 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57746b86db-whs5h"] Jul 06 19:36:21.201071 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:21.201049 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57746b86db-whs5h"] Jul 06 19:36:21.686354 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:21.686329 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="082cf40c-a116-4e8d-a556-8e04a6205e49" path="/var/lib/kubelet/pods/082cf40c-a116-4e8d-a556-8e04a6205e49/volumes" Jul 06 19:36:53.727169 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.727137 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk"] Jul 06 19:36:53.727689 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.727629 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="082cf40c-a116-4e8d-a556-8e04a6205e49" containerName="console" Jul 06 19:36:53.727689 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.727647 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="082cf40c-a116-4e8d-a556-8e04a6205e49" containerName="console" Jul 06 19:36:53.727771 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.727720 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="082cf40c-a116-4e8d-a556-8e04a6205e49" containerName="console" Jul 06 19:36:53.731388 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.731372 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.733811 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.733793 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-94xrv\"" Jul 06 19:36:53.743110 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.743085 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk"] Jul 06 19:36:53.873187 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.873147 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.873187 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.873186 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.873489 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.873211 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.873489 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.873242 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqc6b\" (UniqueName: \"kubernetes.io/projected/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-kube-api-access-rqc6b\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.873489 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.873338 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.873489 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.873380 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.873489 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.873406 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.873489 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.873468 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.873745 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.873498 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.974341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.974269 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.974546 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.974357 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.974546 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.974397 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.974546 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.974426 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqc6b\" (UniqueName: \"kubernetes.io/projected/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-kube-api-access-rqc6b\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.974714 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.974581 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.974714 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.974620 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.974714 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.974647 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.974866 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.974713 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.974866 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.974739 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.974993 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.974970 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.975071 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.975052 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.975165 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.975143 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.975224 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.975160 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.975348 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.975331 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.976820 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.976796 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.976960 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.976940 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.982267 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.982214 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:53.982552 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:53.982531 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqc6b\" (UniqueName: \"kubernetes.io/projected/c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e-kube-api-access-rqc6b\") pod \"maas-default-gateway-openshift-default-56654dcf7-cs4bk\" (UID: \"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:54.041495 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:54.041464 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:54.177059 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:54.177032 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk"] Jul 06 19:36:54.178785 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:36:54.178759 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9c03d15_1eb0_48d0_9d4f_98c8577f6d6e.slice/crio-de4258974734c9f3dd57718057463ea8e4acc3423b1f923a06969ed7b5d2dc6d WatchSource:0}: Error finding container de4258974734c9f3dd57718057463ea8e4acc3423b1f923a06969ed7b5d2dc6d: Status 404 returned error can't find the container with id de4258974734c9f3dd57718057463ea8e4acc3423b1f923a06969ed7b5d2dc6d Jul 06 19:36:54.180740 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:54.180711 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:36:54.180803 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:54.180772 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:36:54.180803 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:54.180799 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:36:54.329069 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:54.328969 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" event={"ID":"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e","Type":"ContainerStarted","Data":"263fec5f14f1c8a269695f569b1ec3553a4ac01ff1138ea52c0ee595d0a07260"} Jul 06 19:36:54.329069 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:54.329030 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" event={"ID":"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e","Type":"ContainerStarted","Data":"de4258974734c9f3dd57718057463ea8e4acc3423b1f923a06969ed7b5d2dc6d"} Jul 06 19:36:54.356226 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:54.356175 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" podStartSLOduration=1.356160941 podStartE2EDuration="1.356160941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:36:54.354037334 +0000 UTC m=+549.221712905" watchObservedRunningTime="2026-07-06 19:36:54.356160941 +0000 UTC m=+549.223836497" Jul 06 19:36:55.042200 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:55.042157 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:55.047058 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:55.047033 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:55.334113 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:55.334023 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:36:55.334810 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:36:55.334790 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:38:19.631271 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:19.631238 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-tnkpv"] Jul 06 19:38:19.634609 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:19.634592 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-tnkpv" Jul 06 19:38:19.637103 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:19.637088 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 19:38:19.637395 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:19.637382 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-b82g9\"" Jul 06 19:38:19.646974 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:19.646953 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-tnkpv"] Jul 06 19:38:19.790000 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:19.789964 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4a687a8-687d-490d-887d-6635e416ae2e-plugins-config-volume\") pod \"payload-pre-processing-7964457696-tnkpv\" (UID: \"a4a687a8-687d-490d-887d-6635e416ae2e\") " pod="openshift-ingress/payload-pre-processing-7964457696-tnkpv" Jul 06 19:38:19.790000 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:19.789999 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jbbx\" (UniqueName: \"kubernetes.io/projected/a4a687a8-687d-490d-887d-6635e416ae2e-kube-api-access-4jbbx\") pod \"payload-pre-processing-7964457696-tnkpv\" (UID: \"a4a687a8-687d-490d-887d-6635e416ae2e\") " pod="openshift-ingress/payload-pre-processing-7964457696-tnkpv" Jul 06 19:38:19.890781 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:19.890702 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4a687a8-687d-490d-887d-6635e416ae2e-plugins-config-volume\") pod \"payload-pre-processing-7964457696-tnkpv\" (UID: \"a4a687a8-687d-490d-887d-6635e416ae2e\") " pod="openshift-ingress/payload-pre-processing-7964457696-tnkpv" Jul 06 19:38:19.890781 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:19.890737 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jbbx\" (UniqueName: \"kubernetes.io/projected/a4a687a8-687d-490d-887d-6635e416ae2e-kube-api-access-4jbbx\") pod \"payload-pre-processing-7964457696-tnkpv\" (UID: \"a4a687a8-687d-490d-887d-6635e416ae2e\") " pod="openshift-ingress/payload-pre-processing-7964457696-tnkpv" Jul 06 19:38:19.891438 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:19.891420 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4a687a8-687d-490d-887d-6635e416ae2e-plugins-config-volume\") pod \"payload-pre-processing-7964457696-tnkpv\" (UID: \"a4a687a8-687d-490d-887d-6635e416ae2e\") " pod="openshift-ingress/payload-pre-processing-7964457696-tnkpv" Jul 06 19:38:19.899463 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:19.899440 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jbbx\" (UniqueName: \"kubernetes.io/projected/a4a687a8-687d-490d-887d-6635e416ae2e-kube-api-access-4jbbx\") pod \"payload-pre-processing-7964457696-tnkpv\" (UID: \"a4a687a8-687d-490d-887d-6635e416ae2e\") " pod="openshift-ingress/payload-pre-processing-7964457696-tnkpv" Jul 06 19:38:19.945459 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:19.945433 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-tnkpv" Jul 06 19:38:20.099529 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:20.099497 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-tnkpv"] Jul 06 19:38:20.100906 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:38:20.100879 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a687a8_687d_490d_887d_6635e416ae2e.slice/crio-60e6adb4446a1fdf89eaa03bd9f97fdd3923c561f1cf321416a37955bcba8636 WatchSource:0}: Error finding container 60e6adb4446a1fdf89eaa03bd9f97fdd3923c561f1cf321416a37955bcba8636: Status 404 returned error can't find the container with id 60e6adb4446a1fdf89eaa03bd9f97fdd3923c561f1cf321416a37955bcba8636 Jul 06 19:38:20.758441 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:20.758396 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-tnkpv" event={"ID":"a4a687a8-687d-490d-887d-6635e416ae2e","Type":"ContainerStarted","Data":"60e6adb4446a1fdf89eaa03bd9f97fdd3923c561f1cf321416a37955bcba8636"} Jul 06 19:38:21.765399 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:21.765371 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-tnkpv" event={"ID":"a4a687a8-687d-490d-887d-6635e416ae2e","Type":"ContainerStarted","Data":"185b5a27a72bd99b6c8c1d28c08d402bfd592bc6ed99e00c721312c3a80eaff3"} Jul 06 19:38:21.765707 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:21.765481 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-tnkpv" Jul 06 19:38:21.785824 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:21.785772 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-tnkpv" podStartSLOduration=1.207662096 podStartE2EDuration="2.785757235s" totalImagesPullingTime="1.578095139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:38:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:38:21.783813131 +0000 UTC m=+636.651488687" watchObservedRunningTime="2026-07-06 19:38:21.785757235 +0000 UTC m=+636.653432817" Jul 06 19:38:26.376450 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:26.376407 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cs4bk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 19:38:26.376861 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:26.376480 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" podUID="c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:38:27.796757 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:27.796721 2561 generic.go:356] "Generic (PLEG): container finished" podID="c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e" containerID="263fec5f14f1c8a269695f569b1ec3553a4ac01ff1138ea52c0ee595d0a07260" exitCode=0 Jul 06 19:38:27.797283 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:27.796752 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" event={"ID":"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e","Type":"ContainerDied","Data":"263fec5f14f1c8a269695f569b1ec3553a4ac01ff1138ea52c0ee595d0a07260"} Jul 06 19:38:27.797283 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:27.797257 2561 scope.go:117] "RemoveContainer" containerID="263fec5f14f1c8a269695f569b1ec3553a4ac01ff1138ea52c0ee595d0a07260" Jul 06 19:38:27.798272 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:27.798246 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:38:27.798376 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:27.798346 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:38:27.798453 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:27.798410 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:38:28.803859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:28.803825 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" event={"ID":"c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e","Type":"ContainerStarted","Data":"acbadc2897777bf93d34a84ac51e7bf2ddb63c5bf04e8d09d19d5ea63926aa21"} Jul 06 19:38:29.042204 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:29.042103 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:38:29.042485 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:29.042433 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cs4bk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 06 19:38:29.042563 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:29.042498 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" podUID="c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 06 19:38:30.042448 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:30.042407 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cs4bk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 06 19:38:30.042935 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:30.042479 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" podUID="c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 06 19:38:31.042151 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:31.042102 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cs4bk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 06 19:38:31.042476 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:31.042445 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" podUID="c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 06 19:38:32.042746 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:32.042705 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cs4bk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 06 19:38:32.043246 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:32.042776 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" podUID="c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 06 19:38:32.772567 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:32.772524 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-7964457696-tnkpv" podUID="a4a687a8-687d-490d-887d-6635e416ae2e" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.33:9004: connect: connection refused" Jul 06 19:38:33.045323 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:33.045230 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:38:33.045723 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:33.045621 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:38:33.046197 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:33.046180 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cs4bk" Jul 06 19:38:42.771698 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:42.771668 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-tnkpv" Jul 06 19:38:42.983198 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:42.983161 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6"] Jul 06 19:38:42.987001 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:42.986979 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:42.989310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:42.989273 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fkdk9\"" Jul 06 19:38:42.989310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:42.989285 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 06 19:38:42.989448 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:42.989400 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 19:38:42.989507 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:42.989404 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 19:38:42.997532 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:42.997512 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6"] Jul 06 19:38:43.106820 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.106728 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdf077ab-76cd-42de-8e90-a47052658963-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.107912 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.107133 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9tw6\" (UniqueName: \"kubernetes.io/projected/bdf077ab-76cd-42de-8e90-a47052658963-kube-api-access-q9tw6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.107912 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.107229 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdf077ab-76cd-42de-8e90-a47052658963-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.113046 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.113016 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdf077ab-76cd-42de-8e90-a47052658963-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.113155 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.113098 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdf077ab-76cd-42de-8e90-a47052658963-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.113370 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.113345 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdf077ab-76cd-42de-8e90-a47052658963-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.214902 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.214860 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdf077ab-76cd-42de-8e90-a47052658963-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.215106 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.214920 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdf077ab-76cd-42de-8e90-a47052658963-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.215106 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.214974 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdf077ab-76cd-42de-8e90-a47052658963-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.215218 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.215117 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdf077ab-76cd-42de-8e90-a47052658963-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.215218 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.215169 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdf077ab-76cd-42de-8e90-a47052658963-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.215218 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.215205 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9tw6\" (UniqueName: \"kubernetes.io/projected/bdf077ab-76cd-42de-8e90-a47052658963-kube-api-access-q9tw6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.215422 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.215392 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdf077ab-76cd-42de-8e90-a47052658963-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.215528 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.215497 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdf077ab-76cd-42de-8e90-a47052658963-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.215661 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.215560 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdf077ab-76cd-42de-8e90-a47052658963-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.217212 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.217187 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdf077ab-76cd-42de-8e90-a47052658963-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.217397 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.217377 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdf077ab-76cd-42de-8e90-a47052658963-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.223636 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.223612 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9tw6\" (UniqueName: \"kubernetes.io/projected/bdf077ab-76cd-42de-8e90-a47052658963-kube-api-access-q9tw6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-68kz6\" (UID: \"bdf077ab-76cd-42de-8e90-a47052658963\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.297284 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.297247 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:43.425952 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.425921 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6"] Jul 06 19:38:43.427704 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:38:43.427681 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdf077ab_76cd_42de_8e90_a47052658963.slice/crio-1a2997bbbdb2cb1efe2481dacab4904904c72e7634eeb66714675db25eebf5b1 WatchSource:0}: Error finding container 1a2997bbbdb2cb1efe2481dacab4904904c72e7634eeb66714675db25eebf5b1: Status 404 returned error can't find the container with id 1a2997bbbdb2cb1efe2481dacab4904904c72e7634eeb66714675db25eebf5b1 Jul 06 19:38:43.429100 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.429084 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:38:43.879550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:43.879456 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" event={"ID":"bdf077ab-76cd-42de-8e90-a47052658963","Type":"ContainerStarted","Data":"1a2997bbbdb2cb1efe2481dacab4904904c72e7634eeb66714675db25eebf5b1"} Jul 06 19:38:44.886122 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:44.886022 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" event={"ID":"bdf077ab-76cd-42de-8e90-a47052658963","Type":"ContainerStarted","Data":"d6818d533646297075d31d6cc0fb91a2508520f7b2e3a35998f5fb94a795c1be"} Jul 06 19:38:44.907851 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:44.907801 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" podStartSLOduration=1.906244707 podStartE2EDuration="2.907783068s" totalImagesPullingTime="1.001538361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:38:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:38:44.90555529 +0000 UTC m=+659.773230845" watchObservedRunningTime="2026-07-06 19:38:44.907783068 +0000 UTC m=+659.775458632" Jul 06 19:38:53.297640 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:53.297590 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:53.298119 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:53.297690 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:53.309720 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:53.309690 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:53.939043 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:53.939013 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-68kz6" Jul 06 19:38:56.102467 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.102412 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8"] Jul 06 19:38:56.106145 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.106117 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.109738 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.109718 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 19:38:56.118196 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.118173 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8"] Jul 06 19:38:56.243525 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.243491 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7ed9752-50da-40ea-ab55-30ec35f86f5f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.243688 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.243549 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tpld\" (UniqueName: \"kubernetes.io/projected/b7ed9752-50da-40ea-ab55-30ec35f86f5f-kube-api-access-5tpld\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.243688 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.243575 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7ed9752-50da-40ea-ab55-30ec35f86f5f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.243688 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.243603 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7ed9752-50da-40ea-ab55-30ec35f86f5f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.243688 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.243621 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7ed9752-50da-40ea-ab55-30ec35f86f5f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.243821 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.243691 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7ed9752-50da-40ea-ab55-30ec35f86f5f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.344677 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.344635 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7ed9752-50da-40ea-ab55-30ec35f86f5f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.344848 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.344727 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tpld\" (UniqueName: \"kubernetes.io/projected/b7ed9752-50da-40ea-ab55-30ec35f86f5f-kube-api-access-5tpld\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.344848 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.344762 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7ed9752-50da-40ea-ab55-30ec35f86f5f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.344848 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.344810 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7ed9752-50da-40ea-ab55-30ec35f86f5f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.344848 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.344840 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7ed9752-50da-40ea-ab55-30ec35f86f5f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.345041 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.344874 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7ed9752-50da-40ea-ab55-30ec35f86f5f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.345041 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.345020 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7ed9752-50da-40ea-ab55-30ec35f86f5f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.345283 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.345263 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7ed9752-50da-40ea-ab55-30ec35f86f5f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.345373 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.345277 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7ed9752-50da-40ea-ab55-30ec35f86f5f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.347100 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.347078 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7ed9752-50da-40ea-ab55-30ec35f86f5f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.347264 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.347246 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7ed9752-50da-40ea-ab55-30ec35f86f5f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.353204 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.353161 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tpld\" (UniqueName: \"kubernetes.io/projected/b7ed9752-50da-40ea-ab55-30ec35f86f5f-kube-api-access-5tpld\") pod \"e2e-trlp-test-simulated-kserve-598466548d-69hl8\" (UID: \"b7ed9752-50da-40ea-ab55-30ec35f86f5f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.415857 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.415814 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:38:56.546514 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.546491 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8"] Jul 06 19:38:56.548321 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:38:56.548283 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ed9752_50da_40ea_ab55_30ec35f86f5f.slice/crio-fdcb42955fc8ba45f850d9aa204dcbe89c79af8faf48ed26dfb037a1e918ae5e WatchSource:0}: Error finding container fdcb42955fc8ba45f850d9aa204dcbe89c79af8faf48ed26dfb037a1e918ae5e: Status 404 returned error can't find the container with id fdcb42955fc8ba45f850d9aa204dcbe89c79af8faf48ed26dfb037a1e918ae5e Jul 06 19:38:56.942724 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.942692 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" event={"ID":"b7ed9752-50da-40ea-ab55-30ec35f86f5f","Type":"ContainerStarted","Data":"d3e6d814711afb34677f05b49528518f670b7101547c70790f4dd56012b5d528"} Jul 06 19:38:56.942888 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.942746 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" event={"ID":"b7ed9752-50da-40ea-ab55-30ec35f86f5f","Type":"ContainerStarted","Data":"fdcb42955fc8ba45f850d9aa204dcbe89c79af8faf48ed26dfb037a1e918ae5e"} Jul 06 19:38:56.962085 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:38:56.962039 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" podStartSLOduration=0.781671218 podStartE2EDuration="962.024189ms" totalImagesPullingTime="180.352971ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:38:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:38:56.960484865 +0000 UTC m=+671.828160423" watchObservedRunningTime="2026-07-06 19:38:56.962024189 +0000 UTC m=+671.829699743" Jul 06 19:39:06.416454 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:06.416418 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:39:06.416929 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:06.416492 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:39:06.428550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:06.428523 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:39:07.001835 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:07.001806 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-69hl8" Jul 06 19:39:21.807518 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.807488 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb"] Jul 06 19:39:21.810888 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.810871 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.813662 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.813644 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 06 19:39:21.825168 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.825144 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb"] Jul 06 19:39:21.887712 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.887682 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7e266be-03d6-45b5-977b-197a93d3d042-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.887859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.887721 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8k94\" (UniqueName: \"kubernetes.io/projected/d7e266be-03d6-45b5-977b-197a93d3d042-kube-api-access-r8k94\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.887859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.887786 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7e266be-03d6-45b5-977b-197a93d3d042-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.887859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.887810 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e266be-03d6-45b5-977b-197a93d3d042-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.887859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.887854 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7e266be-03d6-45b5-977b-197a93d3d042-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.887996 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.887903 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7e266be-03d6-45b5-977b-197a93d3d042-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.989027 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.988997 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7e266be-03d6-45b5-977b-197a93d3d042-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.989206 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.989053 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7e266be-03d6-45b5-977b-197a93d3d042-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.989206 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.989073 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8k94\" (UniqueName: \"kubernetes.io/projected/d7e266be-03d6-45b5-977b-197a93d3d042-kube-api-access-r8k94\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.989206 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.989110 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7e266be-03d6-45b5-977b-197a93d3d042-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.989206 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.989125 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e266be-03d6-45b5-977b-197a93d3d042-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.989206 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.989184 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7e266be-03d6-45b5-977b-197a93d3d042-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.989532 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.989512 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e266be-03d6-45b5-977b-197a93d3d042-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.989576 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.989511 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7e266be-03d6-45b5-977b-197a93d3d042-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.989619 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.989598 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7e266be-03d6-45b5-977b-197a93d3d042-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.991435 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.991414 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7e266be-03d6-45b5-977b-197a93d3d042-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.991588 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.991575 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7e266be-03d6-45b5-977b-197a93d3d042-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:21.997217 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:21.997188 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8k94\" (UniqueName: \"kubernetes.io/projected/d7e266be-03d6-45b5-977b-197a93d3d042-kube-api-access-r8k94\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lz5vb\" (UID: \"d7e266be-03d6-45b5-977b-197a93d3d042\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:22.121670 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:22.121596 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:22.252102 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:22.252073 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb"] Jul 06 19:39:22.255176 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:39:22.255152 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e266be_03d6_45b5_977b_197a93d3d042.slice/crio-160f6d4df889eda9fdc78b34cf86a93da8ce88a9e62ff26f11879afbcc494ffa WatchSource:0}: Error finding container 160f6d4df889eda9fdc78b34cf86a93da8ce88a9e62ff26f11879afbcc494ffa: Status 404 returned error can't find the container with id 160f6d4df889eda9fdc78b34cf86a93da8ce88a9e62ff26f11879afbcc494ffa Jul 06 19:39:23.070720 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:23.070682 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" event={"ID":"d7e266be-03d6-45b5-977b-197a93d3d042","Type":"ContainerStarted","Data":"5d855aa03889b70e1d891d1f3649776135fc93f0035458aa4c7e44179814cff4"} Jul 06 19:39:23.071097 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:23.070773 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" event={"ID":"d7e266be-03d6-45b5-977b-197a93d3d042","Type":"ContainerStarted","Data":"160f6d4df889eda9fdc78b34cf86a93da8ce88a9e62ff26f11879afbcc494ffa"} Jul 06 19:39:23.090252 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:23.090203 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" podStartSLOduration=1.9328865880000001 podStartE2EDuration="2.09018295s" totalImagesPullingTime="157.296362ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:39:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:39:23.088832425 +0000 UTC m=+697.956507982" watchObservedRunningTime="2026-07-06 19:39:23.09018295 +0000 UTC m=+697.957858504" Jul 06 19:39:32.122859 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:32.122821 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:32.123274 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:32.122887 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:32.134920 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:32.134898 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:39:33.128891 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:39:33.128855 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lz5vb" Jul 06 19:54:22.356136 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.356103 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp"] Jul 06 19:54:22.360253 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.360230 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.362620 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.362599 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8f9e41e4-openshift-default-dockercfg-xx224\"" Jul 06 19:54:22.378014 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.377989 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp"] Jul 06 19:54:22.394604 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.394577 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istiod-ca-cert\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.394731 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.394627 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-envoy\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.394731 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.394651 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-data\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.394731 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.394672 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-token\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.394731 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.394700 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-workload-socket\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.394891 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.394753 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-workload-certs\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.394891 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.394777 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-credential-socket\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.394891 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.394830 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-podinfo\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.394891 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.394853 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67pnt\" (UniqueName: \"kubernetes.io/projected/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-kube-api-access-67pnt\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.495723 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.495689 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istiod-ca-cert\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.495906 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.495735 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-envoy\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.495906 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.495757 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-data\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.495906 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.495775 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-token\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.495906 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.495797 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-workload-socket\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.496109 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.495954 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-workload-certs\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.496109 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.496015 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-credential-socket\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.496109 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.496094 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-podinfo\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.496251 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.496123 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67pnt\" (UniqueName: \"kubernetes.io/projected/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-kube-api-access-67pnt\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.496251 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.496209 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-workload-socket\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.496433 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.496406 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-data\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.496497 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.496434 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-credential-socket\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.496497 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.496482 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-workload-certs\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.496565 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.496514 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istiod-ca-cert\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.498142 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.498117 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-envoy\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.498459 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.498441 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-podinfo\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.503846 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.503820 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-token\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.504124 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.504105 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67pnt\" (UniqueName: \"kubernetes.io/projected/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-kube-api-access-67pnt\") pod \"e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.672894 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.672863 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:22.809938 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.809913 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp"] Jul 06 19:54:22.812115 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:54:22.812088 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod928fe20e_958c_4cb5_a9b2_80cbb9fafee3.slice/crio-2eac6c626f3590c0e91de29553e8e13c7b268909823d04158da42b82d8b6d0b1 WatchSource:0}: Error finding container 2eac6c626f3590c0e91de29553e8e13c7b268909823d04158da42b82d8b6d0b1: Status 404 returned error can't find the container with id 2eac6c626f3590c0e91de29553e8e13c7b268909823d04158da42b82d8b6d0b1 Jul 06 19:54:22.814234 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.814192 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:54:22.814360 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.814261 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:54:22.814360 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:22.814313 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:54:23.398861 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:23.398816 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" event={"ID":"928fe20e-958c-4cb5-a9b2-80cbb9fafee3","Type":"ContainerStarted","Data":"b5f5e851b9450b80e45940efc525b73896c5c9da52b566b414c30cd83448f851"} Jul 06 19:54:23.399338 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:23.398884 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" event={"ID":"928fe20e-958c-4cb5-a9b2-80cbb9fafee3","Type":"ContainerStarted","Data":"2eac6c626f3590c0e91de29553e8e13c7b268909823d04158da42b82d8b6d0b1"} Jul 06 19:54:23.419212 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:23.419167 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" podStartSLOduration=1.419150511 podStartE2EDuration="1.419150511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:54:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:54:23.41833234 +0000 UTC m=+1598.286007902" watchObservedRunningTime="2026-07-06 19:54:23.419150511 +0000 UTC m=+1598.286826069" Jul 06 19:54:23.673166 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:23.673132 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:23.677930 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:23.677907 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:24.403599 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:24.403560 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:24.404677 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:24.404650 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:28.951601 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:28.951559 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp"] Jul 06 19:54:28.952209 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:28.951785 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" podUID="928fe20e-958c-4cb5-a9b2-80cbb9fafee3" containerName="istio-proxy" containerID="cri-o://b5f5e851b9450b80e45940efc525b73896c5c9da52b566b414c30cd83448f851" gracePeriod=30 Jul 06 19:54:34.798283 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.798254 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:34.930685 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.930655 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-credential-socket\") pod \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " Jul 06 19:54:34.930863 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.930740 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istiod-ca-cert\") pod \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " Jul 06 19:54:34.930863 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.930783 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-envoy\") pod \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " Jul 06 19:54:34.930863 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.930811 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-data\") pod \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " Jul 06 19:54:34.930863 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.930839 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-workload-socket\") pod \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " Jul 06 19:54:34.931095 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.930864 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67pnt\" (UniqueName: \"kubernetes.io/projected/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-kube-api-access-67pnt\") pod \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " Jul 06 19:54:34.931095 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.930900 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-token\") pod \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " Jul 06 19:54:34.931095 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.930905 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "928fe20e-958c-4cb5-a9b2-80cbb9fafee3" (UID: "928fe20e-958c-4cb5-a9b2-80cbb9fafee3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:54:34.931095 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.930923 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-workload-certs\") pod \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " Jul 06 19:54:34.931095 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.930974 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-podinfo\") pod \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\" (UID: \"928fe20e-958c-4cb5-a9b2-80cbb9fafee3\") " Jul 06 19:54:34.931357 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.931150 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "928fe20e-958c-4cb5-a9b2-80cbb9fafee3" (UID: "928fe20e-958c-4cb5-a9b2-80cbb9fafee3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:54:34.931357 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.931218 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "928fe20e-958c-4cb5-a9b2-80cbb9fafee3" (UID: "928fe20e-958c-4cb5-a9b2-80cbb9fafee3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:54:34.931514 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.931491 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "928fe20e-958c-4cb5-a9b2-80cbb9fafee3" (UID: "928fe20e-958c-4cb5-a9b2-80cbb9fafee3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:54:34.931514 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.931503 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istiod-ca-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:34.931616 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.931524 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-workload-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:34.931616 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.931537 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-credential-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:34.931910 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.931888 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-data" (OuterVolumeSpecName: "istio-data") pod "928fe20e-958c-4cb5-a9b2-80cbb9fafee3" (UID: "928fe20e-958c-4cb5-a9b2-80cbb9fafee3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:54:34.933207 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.933188 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "928fe20e-958c-4cb5-a9b2-80cbb9fafee3" (UID: "928fe20e-958c-4cb5-a9b2-80cbb9fafee3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:54:34.933317 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.933213 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-token" (OuterVolumeSpecName: "istio-token") pod "928fe20e-958c-4cb5-a9b2-80cbb9fafee3" (UID: "928fe20e-958c-4cb5-a9b2-80cbb9fafee3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:54:34.933371 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.933314 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "928fe20e-958c-4cb5-a9b2-80cbb9fafee3" (UID: "928fe20e-958c-4cb5-a9b2-80cbb9fafee3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:54:34.933407 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:34.933366 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-kube-api-access-67pnt" (OuterVolumeSpecName: "kube-api-access-67pnt") pod "928fe20e-958c-4cb5-a9b2-80cbb9fafee3" (UID: "928fe20e-958c-4cb5-a9b2-80cbb9fafee3"). InnerVolumeSpecName "kube-api-access-67pnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:54:35.032016 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.031981 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-envoy\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:35.032016 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.032009 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-data\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:35.032016 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.032019 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67pnt\" (UniqueName: \"kubernetes.io/projected/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-kube-api-access-67pnt\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:35.032243 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.032028 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-token\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:35.032243 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.032037 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-workload-certs\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:35.032243 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.032044 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928fe20e-958c-4cb5-a9b2-80cbb9fafee3-istio-podinfo\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:35.185507 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.185429 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt"] Jul 06 19:54:35.185869 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.185855 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="928fe20e-958c-4cb5-a9b2-80cbb9fafee3" containerName="istio-proxy" Jul 06 19:54:35.185869 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.185869 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="928fe20e-958c-4cb5-a9b2-80cbb9fafee3" containerName="istio-proxy" Jul 06 19:54:35.185972 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.185942 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="928fe20e-958c-4cb5-a9b2-80cbb9fafee3" containerName="istio-proxy" Jul 06 19:54:35.188094 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.188073 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.190482 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.190456 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-363e7ca9-openshift-default-dockercfg-r2xvr\"" Jul 06 19:54:35.202514 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.202492 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt"] Jul 06 19:54:35.233917 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.233884 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-token\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.234077 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.233930 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5tbk\" (UniqueName: \"kubernetes.io/projected/a3ae22f6-a12f-41a1-bb9a-0346e0246957-kube-api-access-j5tbk\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.234077 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.233960 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-envoy\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.234077 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.234024 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-data\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.234077 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.234063 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-workload-certs\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.234248 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.234095 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-podinfo\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.234248 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.234130 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-workload-socket\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.234248 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.234183 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-credential-socket\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.234248 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.234221 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istiod-ca-cert\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.334887 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.334856 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-podinfo\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.334887 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.334893 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-workload-socket\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.335145 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.334922 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-credential-socket\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.335145 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.335049 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istiod-ca-cert\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.335145 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.335132 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-token\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.335337 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.335192 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5tbk\" (UniqueName: \"kubernetes.io/projected/a3ae22f6-a12f-41a1-bb9a-0346e0246957-kube-api-access-j5tbk\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.335337 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.335252 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-envoy\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.335337 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.335323 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-credential-socket\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.335496 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.335369 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-workload-socket\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.335496 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.335438 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-data\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.335593 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.335489 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-workload-certs\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.335777 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.335751 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-data\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.335854 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.335787 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-workload-certs\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.335854 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.335845 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istiod-ca-cert\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.337307 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.337268 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-envoy\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.337439 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.337422 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-podinfo\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.352466 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.352439 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-token\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.353994 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.353973 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5tbk\" (UniqueName: \"kubernetes.io/projected/a3ae22f6-a12f-41a1-bb9a-0346e0246957-kube-api-access-j5tbk\") pod \"e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.464081 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.464000 2561 generic.go:356] "Generic (PLEG): container finished" podID="928fe20e-958c-4cb5-a9b2-80cbb9fafee3" containerID="b5f5e851b9450b80e45940efc525b73896c5c9da52b566b414c30cd83448f851" exitCode=0 Jul 06 19:54:35.464081 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.464054 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" Jul 06 19:54:35.464248 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.464089 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" event={"ID":"928fe20e-958c-4cb5-a9b2-80cbb9fafee3","Type":"ContainerDied","Data":"b5f5e851b9450b80e45940efc525b73896c5c9da52b566b414c30cd83448f851"} Jul 06 19:54:35.464248 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.464135 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp" event={"ID":"928fe20e-958c-4cb5-a9b2-80cbb9fafee3","Type":"ContainerDied","Data":"2eac6c626f3590c0e91de29553e8e13c7b268909823d04158da42b82d8b6d0b1"} Jul 06 19:54:35.464248 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.464154 2561 scope.go:117] "RemoveContainer" containerID="b5f5e851b9450b80e45940efc525b73896c5c9da52b566b414c30cd83448f851" Jul 06 19:54:35.473628 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.473612 2561 scope.go:117] "RemoveContainer" containerID="b5f5e851b9450b80e45940efc525b73896c5c9da52b566b414c30cd83448f851" Jul 06 19:54:35.473867 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:54:35.473846 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5f5e851b9450b80e45940efc525b73896c5c9da52b566b414c30cd83448f851\": container with ID starting with b5f5e851b9450b80e45940efc525b73896c5c9da52b566b414c30cd83448f851 not found: ID does not exist" containerID="b5f5e851b9450b80e45940efc525b73896c5c9da52b566b414c30cd83448f851" Jul 06 19:54:35.473930 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.473876 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5f5e851b9450b80e45940efc525b73896c5c9da52b566b414c30cd83448f851"} err="failed to get container status \"b5f5e851b9450b80e45940efc525b73896c5c9da52b566b414c30cd83448f851\": rpc error: code = NotFound desc = could not find container \"b5f5e851b9450b80e45940efc525b73896c5c9da52b566b414c30cd83448f851\": container with ID starting with b5f5e851b9450b80e45940efc525b73896c5c9da52b566b414c30cd83448f851 not found: ID does not exist" Jul 06 19:54:35.498019 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.497997 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp"] Jul 06 19:54:35.499538 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.499500 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:35.502188 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.502172 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8f9e41e4-openshift-default-7544cffb59-qg7dp"] Jul 06 19:54:35.638371 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.638348 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt"] Jul 06 19:54:35.640633 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:54:35.640606 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3ae22f6_a12f_41a1_bb9a_0346e0246957.slice/crio-d566a04c0d58a60825f2cd19b611aa71f308d4010027b39625feafff502c0580 WatchSource:0}: Error finding container d566a04c0d58a60825f2cd19b611aa71f308d4010027b39625feafff502c0580: Status 404 returned error can't find the container with id d566a04c0d58a60825f2cd19b611aa71f308d4010027b39625feafff502c0580 Jul 06 19:54:35.642647 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.642618 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:54:35.642721 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.642684 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:54:35.642721 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.642713 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:54:35.688553 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:35.688529 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="928fe20e-958c-4cb5-a9b2-80cbb9fafee3" path="/var/lib/kubelet/pods/928fe20e-958c-4cb5-a9b2-80cbb9fafee3/volumes" Jul 06 19:54:36.471746 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:36.471712 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" event={"ID":"a3ae22f6-a12f-41a1-bb9a-0346e0246957","Type":"ContainerStarted","Data":"16dc9b89e65f980aa532e146bdcfe556c05730d425829e5bb2ea2bfff351daac"} Jul 06 19:54:36.472120 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:36.471760 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" event={"ID":"a3ae22f6-a12f-41a1-bb9a-0346e0246957","Type":"ContainerStarted","Data":"d566a04c0d58a60825f2cd19b611aa71f308d4010027b39625feafff502c0580"} Jul 06 19:54:36.497699 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:36.497647 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" podStartSLOduration=1.49762796 podStartE2EDuration="1.49762796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:54:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:54:36.496830516 +0000 UTC m=+1611.364506073" watchObservedRunningTime="2026-07-06 19:54:36.49762796 +0000 UTC m=+1611.365303516" Jul 06 19:54:36.500382 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:36.500348 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:36.505067 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:36.505046 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:37.477098 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:37.477025 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:37.477883 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:37.477861 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:47.530612 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:47.530575 2561 generic.go:356] "Generic (PLEG): container finished" podID="a3ae22f6-a12f-41a1-bb9a-0346e0246957" containerID="16dc9b89e65f980aa532e146bdcfe556c05730d425829e5bb2ea2bfff351daac" exitCode=0 Jul 06 19:54:47.531030 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:47.530648 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" event={"ID":"a3ae22f6-a12f-41a1-bb9a-0346e0246957","Type":"ContainerDied","Data":"16dc9b89e65f980aa532e146bdcfe556c05730d425829e5bb2ea2bfff351daac"} Jul 06 19:54:47.531079 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:47.531070 2561 scope.go:117] "RemoveContainer" containerID="16dc9b89e65f980aa532e146bdcfe556c05730d425829e5bb2ea2bfff351daac" Jul 06 19:54:47.532043 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:47.532020 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:54:47.532091 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:47.532080 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:54:47.532125 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:47.532110 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:54:47.638036 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:47.638008 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt"] Jul 06 19:54:48.537355 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:48.537318 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" event={"ID":"a3ae22f6-a12f-41a1-bb9a-0346e0246957","Type":"ContainerStarted","Data":"96e0bd3dd6f456dbce8bab87fbdf255552f979d676ab96bfdf4192b5247be855"} Jul 06 19:54:48.537803 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:48.537442 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" podUID="a3ae22f6-a12f-41a1-bb9a-0346e0246957" containerName="istio-proxy" containerID="cri-o://96e0bd3dd6f456dbce8bab87fbdf255552f979d676ab96bfdf4192b5247be855" gracePeriod=30 Jul 06 19:54:52.477177 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:52.477129 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:53.799737 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.799678 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:53.922025 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.921986 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5tbk\" (UniqueName: \"kubernetes.io/projected/a3ae22f6-a12f-41a1-bb9a-0346e0246957-kube-api-access-j5tbk\") pod \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " Jul 06 19:54:53.922227 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.922039 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-data\") pod \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " Jul 06 19:54:53.922227 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.922059 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-workload-socket\") pod \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " Jul 06 19:54:53.922227 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.922123 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-credential-socket\") pod \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " Jul 06 19:54:53.922227 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.922153 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-token\") pod \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " Jul 06 19:54:53.922227 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.922204 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istiod-ca-cert\") pod \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " Jul 06 19:54:53.922550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.922231 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-podinfo\") pod \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " Jul 06 19:54:53.922550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.922276 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-workload-certs\") pod \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " Jul 06 19:54:53.922550 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.922524 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a3ae22f6-a12f-41a1-bb9a-0346e0246957" (UID: "a3ae22f6-a12f-41a1-bb9a-0346e0246957"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:54:53.922704 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.922541 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a3ae22f6-a12f-41a1-bb9a-0346e0246957" (UID: "a3ae22f6-a12f-41a1-bb9a-0346e0246957"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:54:53.922704 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.922552 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a3ae22f6-a12f-41a1-bb9a-0346e0246957" (UID: "a3ae22f6-a12f-41a1-bb9a-0346e0246957"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:54:53.922704 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.922605 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-envoy\") pod \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\" (UID: \"a3ae22f6-a12f-41a1-bb9a-0346e0246957\") " Jul 06 19:54:53.922853 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.922695 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a3ae22f6-a12f-41a1-bb9a-0346e0246957" (UID: "a3ae22f6-a12f-41a1-bb9a-0346e0246957"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:54:53.923023 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.923000 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-credential-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:53.923100 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.923032 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istiod-ca-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:53.923100 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.923043 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-workload-certs\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:53.923100 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.923051 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-workload-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:53.923100 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.923000 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-data" (OuterVolumeSpecName: "istio-data") pod "a3ae22f6-a12f-41a1-bb9a-0346e0246957" (UID: "a3ae22f6-a12f-41a1-bb9a-0346e0246957"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:54:53.924450 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.924417 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-token" (OuterVolumeSpecName: "istio-token") pod "a3ae22f6-a12f-41a1-bb9a-0346e0246957" (UID: "a3ae22f6-a12f-41a1-bb9a-0346e0246957"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:54:53.924631 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.924612 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a3ae22f6-a12f-41a1-bb9a-0346e0246957" (UID: "a3ae22f6-a12f-41a1-bb9a-0346e0246957"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:54:53.924789 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.924768 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a3ae22f6-a12f-41a1-bb9a-0346e0246957" (UID: "a3ae22f6-a12f-41a1-bb9a-0346e0246957"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:54:53.924849 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.924774 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3ae22f6-a12f-41a1-bb9a-0346e0246957-kube-api-access-j5tbk" (OuterVolumeSpecName: "kube-api-access-j5tbk") pod "a3ae22f6-a12f-41a1-bb9a-0346e0246957" (UID: "a3ae22f6-a12f-41a1-bb9a-0346e0246957"). InnerVolumeSpecName "kube-api-access-j5tbk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:54:53.958256 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.958229 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82"] Jul 06 19:54:53.958664 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.958647 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3ae22f6-a12f-41a1-bb9a-0346e0246957" containerName="istio-proxy" Jul 06 19:54:53.958664 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.958664 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3ae22f6-a12f-41a1-bb9a-0346e0246957" containerName="istio-proxy" Jul 06 19:54:53.958809 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.958684 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3ae22f6-a12f-41a1-bb9a-0346e0246957" containerName="istio-proxy" Jul 06 19:54:53.958809 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.958692 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3ae22f6-a12f-41a1-bb9a-0346e0246957" containerName="istio-proxy" Jul 06 19:54:53.958918 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.958815 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3ae22f6-a12f-41a1-bb9a-0346e0246957" containerName="istio-proxy" Jul 06 19:54:53.958918 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.958830 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3ae22f6-a12f-41a1-bb9a-0346e0246957" containerName="istio-proxy" Jul 06 19:54:53.962218 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.962200 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:53.964742 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.964723 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-e1149fa3-openshift-default-dockercfg-zc8xb\"" Jul 06 19:54:53.980360 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:53.980334 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82"] Jul 06 19:54:54.023822 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.023786 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-data\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:54.023822 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.023815 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-token\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:54.023822 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.023824 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-podinfo\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:54.023822 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.023832 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3ae22f6-a12f-41a1-bb9a-0346e0246957-istio-envoy\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:54.024088 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.023840 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5tbk\" (UniqueName: \"kubernetes.io/projected/a3ae22f6-a12f-41a1-bb9a-0346e0246957-kube-api-access-j5tbk\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:54:54.125403 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.125308 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-workload-certs\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.125403 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.125376 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-envoy\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.125595 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.125408 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-credential-socket\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.125595 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.125429 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-token\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.125595 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.125543 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-workload-socket\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.125691 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.125593 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istiod-ca-cert\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.125691 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.125623 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-data\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.125691 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.125650 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-859ph\" (UniqueName: \"kubernetes.io/projected/b4045395-25b7-44f1-b2eb-cb8fa61272ce-kube-api-access-859ph\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.125691 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.125675 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-podinfo\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.226758 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.226718 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istiod-ca-cert\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.226758 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.226766 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-data\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.226966 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.226790 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-859ph\" (UniqueName: \"kubernetes.io/projected/b4045395-25b7-44f1-b2eb-cb8fa61272ce-kube-api-access-859ph\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.226966 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.226902 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-podinfo\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.227049 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.226967 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-workload-certs\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.227049 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.227026 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-envoy\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.227152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.227079 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-credential-socket\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.227152 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.227121 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-token\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.227259 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.227218 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-workload-socket\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.227259 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.227121 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-data\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.227422 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.227396 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-workload-certs\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.227545 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.227522 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-credential-socket\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.227598 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.227560 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-workload-socket\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.227648 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.227616 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istiod-ca-cert\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.229320 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.229283 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-envoy\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.229544 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.229518 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-podinfo\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.242706 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.242679 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-token\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.243673 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.243652 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-859ph\" (UniqueName: \"kubernetes.io/projected/b4045395-25b7-44f1-b2eb-cb8fa61272ce-kube-api-access-859ph\") pod \"e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.272715 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.272678 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:54.416869 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.416839 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82"] Jul 06 19:54:54.417610 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:54:54.417587 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4045395_25b7_44f1_b2eb_cb8fa61272ce.slice/crio-a97928dd68773e8a4aadd736f52acd9dbc6f127b66e56721d2a97351d04d3776 WatchSource:0}: Error finding container a97928dd68773e8a4aadd736f52acd9dbc6f127b66e56721d2a97351d04d3776: Status 404 returned error can't find the container with id a97928dd68773e8a4aadd736f52acd9dbc6f127b66e56721d2a97351d04d3776 Jul 06 19:54:54.419674 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.419645 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:54:54.419740 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.419732 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:54:54.419774 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.419763 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:54:54.569370 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.569280 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" event={"ID":"b4045395-25b7-44f1-b2eb-cb8fa61272ce","Type":"ContainerStarted","Data":"e2a0349a0b8e380b9ae9d6ec25c49d5b1a5c14e2315008c892f3ac49f6fd1637"} Jul 06 19:54:54.569558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.569398 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" event={"ID":"b4045395-25b7-44f1-b2eb-cb8fa61272ce","Type":"ContainerStarted","Data":"a97928dd68773e8a4aadd736f52acd9dbc6f127b66e56721d2a97351d04d3776"} Jul 06 19:54:54.571172 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.571143 2561 generic.go:356] "Generic (PLEG): container finished" podID="a3ae22f6-a12f-41a1-bb9a-0346e0246957" containerID="96e0bd3dd6f456dbce8bab87fbdf255552f979d676ab96bfdf4192b5247be855" exitCode=0 Jul 06 19:54:54.571338 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.571195 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" Jul 06 19:54:54.571338 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.571199 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" event={"ID":"a3ae22f6-a12f-41a1-bb9a-0346e0246957","Type":"ContainerDied","Data":"96e0bd3dd6f456dbce8bab87fbdf255552f979d676ab96bfdf4192b5247be855"} Jul 06 19:54:54.571338 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.571334 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt" event={"ID":"a3ae22f6-a12f-41a1-bb9a-0346e0246957","Type":"ContainerDied","Data":"d566a04c0d58a60825f2cd19b611aa71f308d4010027b39625feafff502c0580"} Jul 06 19:54:54.571500 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.571360 2561 scope.go:117] "RemoveContainer" containerID="96e0bd3dd6f456dbce8bab87fbdf255552f979d676ab96bfdf4192b5247be855" Jul 06 19:54:54.582019 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.582001 2561 scope.go:117] "RemoveContainer" containerID="16dc9b89e65f980aa532e146bdcfe556c05730d425829e5bb2ea2bfff351daac" Jul 06 19:54:54.590958 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.590937 2561 scope.go:117] "RemoveContainer" containerID="96e0bd3dd6f456dbce8bab87fbdf255552f979d676ab96bfdf4192b5247be855" Jul 06 19:54:54.591220 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.591182 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" podStartSLOduration=1.5911676049999999 podStartE2EDuration="1.591167605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:54:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:54:54.588265805 +0000 UTC m=+1629.455941365" watchObservedRunningTime="2026-07-06 19:54:54.591167605 +0000 UTC m=+1629.458843160" Jul 06 19:54:54.591337 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:54:54.591209 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96e0bd3dd6f456dbce8bab87fbdf255552f979d676ab96bfdf4192b5247be855\": container with ID starting with 96e0bd3dd6f456dbce8bab87fbdf255552f979d676ab96bfdf4192b5247be855 not found: ID does not exist" containerID="96e0bd3dd6f456dbce8bab87fbdf255552f979d676ab96bfdf4192b5247be855" Jul 06 19:54:54.591409 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.591349 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96e0bd3dd6f456dbce8bab87fbdf255552f979d676ab96bfdf4192b5247be855"} err="failed to get container status \"96e0bd3dd6f456dbce8bab87fbdf255552f979d676ab96bfdf4192b5247be855\": rpc error: code = NotFound desc = could not find container \"96e0bd3dd6f456dbce8bab87fbdf255552f979d676ab96bfdf4192b5247be855\": container with ID starting with 96e0bd3dd6f456dbce8bab87fbdf255552f979d676ab96bfdf4192b5247be855 not found: ID does not exist" Jul 06 19:54:54.591474 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.591398 2561 scope.go:117] "RemoveContainer" containerID="16dc9b89e65f980aa532e146bdcfe556c05730d425829e5bb2ea2bfff351daac" Jul 06 19:54:54.591649 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:54:54.591624 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16dc9b89e65f980aa532e146bdcfe556c05730d425829e5bb2ea2bfff351daac\": container with ID starting with 16dc9b89e65f980aa532e146bdcfe556c05730d425829e5bb2ea2bfff351daac not found: ID does not exist" containerID="16dc9b89e65f980aa532e146bdcfe556c05730d425829e5bb2ea2bfff351daac" Jul 06 19:54:54.591764 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.591657 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16dc9b89e65f980aa532e146bdcfe556c05730d425829e5bb2ea2bfff351daac"} err="failed to get container status \"16dc9b89e65f980aa532e146bdcfe556c05730d425829e5bb2ea2bfff351daac\": rpc error: code = NotFound desc = could not find container \"16dc9b89e65f980aa532e146bdcfe556c05730d425829e5bb2ea2bfff351daac\": container with ID starting with 16dc9b89e65f980aa532e146bdcfe556c05730d425829e5bb2ea2bfff351daac not found: ID does not exist" Jul 06 19:54:54.603615 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.603589 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt"] Jul 06 19:54:54.609869 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:54.609844 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-363e7ca9-openshift-default-6f499b5d57-fmzgt"] Jul 06 19:54:55.273310 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:55.273260 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:55.280236 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:55.280204 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:55.580600 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:55.580518 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:55.581335 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:55.581314 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:54:55.688279 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:55.688250 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3ae22f6-a12f-41a1-bb9a-0346e0246957" path="/var/lib/kubelet/pods/a3ae22f6-a12f-41a1-bb9a-0346e0246957/volumes" Jul 06 19:54:59.741648 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:59.741604 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82"] Jul 06 19:54:59.742164 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:54:59.741835 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" podUID="b4045395-25b7-44f1-b2eb-cb8fa61272ce" containerName="istio-proxy" containerID="cri-o://e2a0349a0b8e380b9ae9d6ec25c49d5b1a5c14e2315008c892f3ac49f6fd1637" gracePeriod=30 Jul 06 19:55:06.497235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.497208 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:55:06.549705 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.549638 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-token\") pod \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " Jul 06 19:55:06.549705 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.549682 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-podinfo\") pod \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " Jul 06 19:55:06.549922 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.549721 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-credential-socket\") pod \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " Jul 06 19:55:06.549922 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.549739 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-envoy\") pod \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " Jul 06 19:55:06.549922 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.549759 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-workload-certs\") pod \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " Jul 06 19:55:06.549922 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.549799 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-data\") pod \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " Jul 06 19:55:06.549922 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.549858 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-859ph\" (UniqueName: \"kubernetes.io/projected/b4045395-25b7-44f1-b2eb-cb8fa61272ce-kube-api-access-859ph\") pod \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " Jul 06 19:55:06.549922 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.549894 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-workload-socket\") pod \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " Jul 06 19:55:06.550239 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.549976 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istiod-ca-cert\") pod \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\" (UID: \"b4045395-25b7-44f1-b2eb-cb8fa61272ce\") " Jul 06 19:55:06.550239 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.550067 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b4045395-25b7-44f1-b2eb-cb8fa61272ce" (UID: "b4045395-25b7-44f1-b2eb-cb8fa61272ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:55:06.550366 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.550256 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b4045395-25b7-44f1-b2eb-cb8fa61272ce" (UID: "b4045395-25b7-44f1-b2eb-cb8fa61272ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:55:06.550457 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.550407 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b4045395-25b7-44f1-b2eb-cb8fa61272ce" (UID: "b4045395-25b7-44f1-b2eb-cb8fa61272ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:55:06.550526 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.550508 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-credential-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:55:06.550599 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.550531 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-workload-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:55:06.550599 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.550551 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b4045395-25b7-44f1-b2eb-cb8fa61272ce" (UID: "b4045395-25b7-44f1-b2eb-cb8fa61272ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:55:06.550811 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.550794 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-data" (OuterVolumeSpecName: "istio-data") pod "b4045395-25b7-44f1-b2eb-cb8fa61272ce" (UID: "b4045395-25b7-44f1-b2eb-cb8fa61272ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:55:06.552170 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.552146 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-token" (OuterVolumeSpecName: "istio-token") pod "b4045395-25b7-44f1-b2eb-cb8fa61272ce" (UID: "b4045395-25b7-44f1-b2eb-cb8fa61272ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:55:06.552350 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.552328 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b4045395-25b7-44f1-b2eb-cb8fa61272ce" (UID: "b4045395-25b7-44f1-b2eb-cb8fa61272ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:55:06.552416 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.552341 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b4045395-25b7-44f1-b2eb-cb8fa61272ce" (UID: "b4045395-25b7-44f1-b2eb-cb8fa61272ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:55:06.552558 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.552541 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4045395-25b7-44f1-b2eb-cb8fa61272ce-kube-api-access-859ph" (OuterVolumeSpecName: "kube-api-access-859ph") pod "b4045395-25b7-44f1-b2eb-cb8fa61272ce" (UID: "b4045395-25b7-44f1-b2eb-cb8fa61272ce"). InnerVolumeSpecName "kube-api-access-859ph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:55:06.637227 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.637197 2561 generic.go:356] "Generic (PLEG): container finished" podID="b4045395-25b7-44f1-b2eb-cb8fa61272ce" containerID="e2a0349a0b8e380b9ae9d6ec25c49d5b1a5c14e2315008c892f3ac49f6fd1637" exitCode=0 Jul 06 19:55:06.637371 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.637243 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" Jul 06 19:55:06.637371 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.637279 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" event={"ID":"b4045395-25b7-44f1-b2eb-cb8fa61272ce","Type":"ContainerDied","Data":"e2a0349a0b8e380b9ae9d6ec25c49d5b1a5c14e2315008c892f3ac49f6fd1637"} Jul 06 19:55:06.637371 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.637346 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82" event={"ID":"b4045395-25b7-44f1-b2eb-cb8fa61272ce","Type":"ContainerDied","Data":"a97928dd68773e8a4aadd736f52acd9dbc6f127b66e56721d2a97351d04d3776"} Jul 06 19:55:06.637371 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.637371 2561 scope.go:117] "RemoveContainer" containerID="e2a0349a0b8e380b9ae9d6ec25c49d5b1a5c14e2315008c892f3ac49f6fd1637" Jul 06 19:55:06.646730 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.646713 2561 scope.go:117] "RemoveContainer" containerID="e2a0349a0b8e380b9ae9d6ec25c49d5b1a5c14e2315008c892f3ac49f6fd1637" Jul 06 19:55:06.646995 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:55:06.646980 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2a0349a0b8e380b9ae9d6ec25c49d5b1a5c14e2315008c892f3ac49f6fd1637\": container with ID starting with e2a0349a0b8e380b9ae9d6ec25c49d5b1a5c14e2315008c892f3ac49f6fd1637 not found: ID does not exist" containerID="e2a0349a0b8e380b9ae9d6ec25c49d5b1a5c14e2315008c892f3ac49f6fd1637" Jul 06 19:55:06.647064 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.647002 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2a0349a0b8e380b9ae9d6ec25c49d5b1a5c14e2315008c892f3ac49f6fd1637"} err="failed to get container status \"e2a0349a0b8e380b9ae9d6ec25c49d5b1a5c14e2315008c892f3ac49f6fd1637\": rpc error: code = NotFound desc = could not find container \"e2a0349a0b8e380b9ae9d6ec25c49d5b1a5c14e2315008c892f3ac49f6fd1637\": container with ID starting with e2a0349a0b8e380b9ae9d6ec25c49d5b1a5c14e2315008c892f3ac49f6fd1637 not found: ID does not exist" Jul 06 19:55:06.651822 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.651800 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-token\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:55:06.651890 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.651829 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-podinfo\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:55:06.651890 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.651842 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-envoy\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:55:06.651890 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.651855 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-workload-certs\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:55:06.651890 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.651866 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istio-data\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:55:06.651890 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.651877 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-859ph\" (UniqueName: \"kubernetes.io/projected/b4045395-25b7-44f1-b2eb-cb8fa61272ce-kube-api-access-859ph\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:55:06.652042 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.651892 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4045395-25b7-44f1-b2eb-cb8fa61272ce-istiod-ca-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:55:06.666805 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.666784 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82"] Jul 06 19:55:06.671524 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:06.671505 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-e1149fa3-openshift-default-56c58c76ff-ntm82"] Jul 06 19:55:07.686462 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:55:07.686422 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4045395-25b7-44f1-b2eb-cb8fa61272ce" path="/var/lib/kubelet/pods/b4045395-25b7-44f1-b2eb-cb8fa61272ce/volumes" Jul 06 19:56:41.722890 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.722820 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6"] Jul 06 19:56:41.723377 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.723224 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4045395-25b7-44f1-b2eb-cb8fa61272ce" containerName="istio-proxy" Jul 06 19:56:41.723377 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.723235 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4045395-25b7-44f1-b2eb-cb8fa61272ce" containerName="istio-proxy" Jul 06 19:56:41.723377 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.723285 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4045395-25b7-44f1-b2eb-cb8fa61272ce" containerName="istio-proxy" Jul 06 19:56:41.726513 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.726498 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.729188 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.729167 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7fd56226-openshift-default-dockercfg-jhd28\"" Jul 06 19:56:41.747807 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.747788 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6"] Jul 06 19:56:41.863374 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.863345 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-credential-socket\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.863541 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.863383 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-podinfo\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.863541 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.863423 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-data\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.863541 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.863514 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-workload-socket\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.863541 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.863533 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-workload-certs\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.863701 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.863586 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-envoy\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.863701 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.863617 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-token\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.863701 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.863651 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6261b27-2db7-466a-b40d-8a09805b31f5-istiod-ca-cert\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.863808 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.863710 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8mqp\" (UniqueName: \"kubernetes.io/projected/b6261b27-2db7-466a-b40d-8a09805b31f5-kube-api-access-v8mqp\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.964561 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.964528 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-envoy\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.964561 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.964566 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-token\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.964780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.964595 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6261b27-2db7-466a-b40d-8a09805b31f5-istiod-ca-cert\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.964780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.964647 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8mqp\" (UniqueName: \"kubernetes.io/projected/b6261b27-2db7-466a-b40d-8a09805b31f5-kube-api-access-v8mqp\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.964780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.964668 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-credential-socket\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.964780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.964695 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-podinfo\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.964780 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.964731 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-data\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.965028 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.964822 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-workload-socket\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.965028 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.964851 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-workload-certs\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.965129 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.965070 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-credential-socket\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.965190 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.965155 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-data\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.965273 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.965251 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-workload-certs\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.965407 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.965384 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-workload-socket\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.965462 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.965441 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6261b27-2db7-466a-b40d-8a09805b31f5-istiod-ca-cert\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.966765 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.966746 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-envoy\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.967172 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.967154 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-podinfo\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.973367 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.973275 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-token\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:41.973448 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:41.973369 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8mqp\" (UniqueName: \"kubernetes.io/projected/b6261b27-2db7-466a-b40d-8a09805b31f5-kube-api-access-v8mqp\") pod \"e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:42.037078 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:42.037051 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:42.180341 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:42.180312 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6"] Jul 06 19:56:42.181815 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:56:42.181789 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6261b27_2db7_466a_b40d_8a09805b31f5.slice/crio-548dd9fb2b149a1de2bf3b6cbc39810291ac1ccec1c6adeb15785472963db810 WatchSource:0}: Error finding container 548dd9fb2b149a1de2bf3b6cbc39810291ac1ccec1c6adeb15785472963db810: Status 404 returned error can't find the container with id 548dd9fb2b149a1de2bf3b6cbc39810291ac1ccec1c6adeb15785472963db810 Jul 06 19:56:42.184074 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:42.184046 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:56:42.184129 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:42.184112 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:56:42.184165 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:42.184139 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:56:43.103253 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:43.103216 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" event={"ID":"b6261b27-2db7-466a-b40d-8a09805b31f5","Type":"ContainerStarted","Data":"32a448a28e8c6b429436483a43730290030ac7a996744c82b59a79a81e354d56"} Jul 06 19:56:43.103744 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:43.103266 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" event={"ID":"b6261b27-2db7-466a-b40d-8a09805b31f5","Type":"ContainerStarted","Data":"548dd9fb2b149a1de2bf3b6cbc39810291ac1ccec1c6adeb15785472963db810"} Jul 06 19:56:43.128521 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:43.128479 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" podStartSLOduration=2.128463528 podStartE2EDuration="2.128463528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:43.126467272 +0000 UTC m=+1737.994142842" watchObservedRunningTime="2026-07-06 19:56:43.128463528 +0000 UTC m=+1737.996139083" Jul 06 19:56:44.037738 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:44.037705 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:44.042425 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:44.042398 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:44.108643 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:44.108615 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:44.109776 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:44.109760 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:58.183549 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:58.183511 2561 generic.go:356] "Generic (PLEG): container finished" podID="b6261b27-2db7-466a-b40d-8a09805b31f5" containerID="32a448a28e8c6b429436483a43730290030ac7a996744c82b59a79a81e354d56" exitCode=0 Jul 06 19:56:58.183974 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:58.183572 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" event={"ID":"b6261b27-2db7-466a-b40d-8a09805b31f5","Type":"ContainerDied","Data":"32a448a28e8c6b429436483a43730290030ac7a996744c82b59a79a81e354d56"} Jul 06 19:56:58.184019 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:58.183996 2561 scope.go:117] "RemoveContainer" containerID="32a448a28e8c6b429436483a43730290030ac7a996744c82b59a79a81e354d56" Jul 06 19:56:58.184990 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:58.184958 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:56:58.185101 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:58.185046 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:56:58.185101 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:58.185095 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:56:59.108808 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:59.108773 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:56:59.190694 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:56:59.190656 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" event={"ID":"b6261b27-2db7-466a-b40d-8a09805b31f5","Type":"ContainerStarted","Data":"cd5853ca1e948fa25da3f3788ae3095026cedfb0d1f20dc538833f7b62d1da4a"} Jul 06 19:57:00.038247 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:00.038209 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:57:00.039608 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:00.039559 2561 patch_prober.go:28] interesting pod/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 06 19:57:00.039820 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:00.039791 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" podUID="b6261b27-2db7-466a-b40d-8a09805b31f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 06 19:57:01.038549 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:01.038510 2561 patch_prober.go:28] interesting pod/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 06 19:57:01.039124 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:01.039092 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" podUID="b6261b27-2db7-466a-b40d-8a09805b31f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 06 19:57:02.052317 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:02.050365 2561 patch_prober.go:28] interesting pod/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 06 19:57:02.052317 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:02.050431 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" podUID="b6261b27-2db7-466a-b40d-8a09805b31f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 06 19:57:03.038613 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:03.038553 2561 patch_prober.go:28] interesting pod/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 06 19:57:03.038972 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:03.038917 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" podUID="b6261b27-2db7-466a-b40d-8a09805b31f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 06 19:57:03.774438 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:03.774403 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6"] Jul 06 19:57:03.774887 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:03.774665 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" podUID="b6261b27-2db7-466a-b40d-8a09805b31f5" containerName="istio-proxy" containerID="cri-o://cd5853ca1e948fa25da3f3788ae3095026cedfb0d1f20dc538833f7b62d1da4a" gracePeriod=30 Jul 06 19:57:09.023186 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.023163 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:57:09.136220 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.136135 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-workload-certs\") pod \"b6261b27-2db7-466a-b40d-8a09805b31f5\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " Jul 06 19:57:09.136220 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.136174 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8mqp\" (UniqueName: \"kubernetes.io/projected/b6261b27-2db7-466a-b40d-8a09805b31f5-kube-api-access-v8mqp\") pod \"b6261b27-2db7-466a-b40d-8a09805b31f5\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " Jul 06 19:57:09.136220 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.136201 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-workload-socket\") pod \"b6261b27-2db7-466a-b40d-8a09805b31f5\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " Jul 06 19:57:09.136220 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.136222 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-token\") pod \"b6261b27-2db7-466a-b40d-8a09805b31f5\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " Jul 06 19:57:09.136568 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.136319 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6261b27-2db7-466a-b40d-8a09805b31f5-istiod-ca-cert\") pod \"b6261b27-2db7-466a-b40d-8a09805b31f5\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " Jul 06 19:57:09.136568 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.136362 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-envoy\") pod \"b6261b27-2db7-466a-b40d-8a09805b31f5\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " Jul 06 19:57:09.136568 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.136388 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-podinfo\") pod \"b6261b27-2db7-466a-b40d-8a09805b31f5\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " Jul 06 19:57:09.136568 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.136415 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-credential-socket\") pod \"b6261b27-2db7-466a-b40d-8a09805b31f5\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " Jul 06 19:57:09.136568 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.136447 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-data\") pod \"b6261b27-2db7-466a-b40d-8a09805b31f5\" (UID: \"b6261b27-2db7-466a-b40d-8a09805b31f5\") " Jul 06 19:57:09.136568 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.136504 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b6261b27-2db7-466a-b40d-8a09805b31f5" (UID: "b6261b27-2db7-466a-b40d-8a09805b31f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:57:09.136568 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.136512 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b6261b27-2db7-466a-b40d-8a09805b31f5" (UID: "b6261b27-2db7-466a-b40d-8a09805b31f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:57:09.136913 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.136872 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6261b27-2db7-466a-b40d-8a09805b31f5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b6261b27-2db7-466a-b40d-8a09805b31f5" (UID: "b6261b27-2db7-466a-b40d-8a09805b31f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:57:09.136913 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.136890 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-workload-certs\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:57:09.136913 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.136908 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-workload-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:57:09.137147 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.137115 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b6261b27-2db7-466a-b40d-8a09805b31f5" (UID: "b6261b27-2db7-466a-b40d-8a09805b31f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:57:09.137603 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.137586 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-data" (OuterVolumeSpecName: "istio-data") pod "b6261b27-2db7-466a-b40d-8a09805b31f5" (UID: "b6261b27-2db7-466a-b40d-8a09805b31f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:57:09.138946 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.138916 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-token" (OuterVolumeSpecName: "istio-token") pod "b6261b27-2db7-466a-b40d-8a09805b31f5" (UID: "b6261b27-2db7-466a-b40d-8a09805b31f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:57:09.139065 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.138953 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6261b27-2db7-466a-b40d-8a09805b31f5-kube-api-access-v8mqp" (OuterVolumeSpecName: "kube-api-access-v8mqp") pod "b6261b27-2db7-466a-b40d-8a09805b31f5" (UID: "b6261b27-2db7-466a-b40d-8a09805b31f5"). InnerVolumeSpecName "kube-api-access-v8mqp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:57:09.139065 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.138984 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b6261b27-2db7-466a-b40d-8a09805b31f5" (UID: "b6261b27-2db7-466a-b40d-8a09805b31f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:57:09.139065 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.138990 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b6261b27-2db7-466a-b40d-8a09805b31f5" (UID: "b6261b27-2db7-466a-b40d-8a09805b31f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:57:09.237720 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.237692 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6261b27-2db7-466a-b40d-8a09805b31f5-istiod-ca-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:57:09.237720 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.237716 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-envoy\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:57:09.237720 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.237724 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-podinfo\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:57:09.237941 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.237733 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-credential-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:57:09.237941 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.237742 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-data\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:57:09.237941 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.237751 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8mqp\" (UniqueName: \"kubernetes.io/projected/b6261b27-2db7-466a-b40d-8a09805b31f5-kube-api-access-v8mqp\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:57:09.237941 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.237759 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6261b27-2db7-466a-b40d-8a09805b31f5-istio-token\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:57:09.243114 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.243085 2561 generic.go:356] "Generic (PLEG): container finished" podID="b6261b27-2db7-466a-b40d-8a09805b31f5" containerID="cd5853ca1e948fa25da3f3788ae3095026cedfb0d1f20dc538833f7b62d1da4a" exitCode=0 Jul 06 19:57:09.243232 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.243130 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" Jul 06 19:57:09.243232 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.243167 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" event={"ID":"b6261b27-2db7-466a-b40d-8a09805b31f5","Type":"ContainerDied","Data":"cd5853ca1e948fa25da3f3788ae3095026cedfb0d1f20dc538833f7b62d1da4a"} Jul 06 19:57:09.243232 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.243218 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6" event={"ID":"b6261b27-2db7-466a-b40d-8a09805b31f5","Type":"ContainerDied","Data":"548dd9fb2b149a1de2bf3b6cbc39810291ac1ccec1c6adeb15785472963db810"} Jul 06 19:57:09.243372 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.243239 2561 scope.go:117] "RemoveContainer" containerID="cd5853ca1e948fa25da3f3788ae3095026cedfb0d1f20dc538833f7b62d1da4a" Jul 06 19:57:09.252740 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.252724 2561 scope.go:117] "RemoveContainer" containerID="32a448a28e8c6b429436483a43730290030ac7a996744c82b59a79a81e354d56" Jul 06 19:57:09.260329 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.260311 2561 scope.go:117] "RemoveContainer" containerID="cd5853ca1e948fa25da3f3788ae3095026cedfb0d1f20dc538833f7b62d1da4a" Jul 06 19:57:09.260603 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:57:09.260582 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd5853ca1e948fa25da3f3788ae3095026cedfb0d1f20dc538833f7b62d1da4a\": container with ID starting with cd5853ca1e948fa25da3f3788ae3095026cedfb0d1f20dc538833f7b62d1da4a not found: ID does not exist" containerID="cd5853ca1e948fa25da3f3788ae3095026cedfb0d1f20dc538833f7b62d1da4a" Jul 06 19:57:09.260675 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.260615 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd5853ca1e948fa25da3f3788ae3095026cedfb0d1f20dc538833f7b62d1da4a"} err="failed to get container status \"cd5853ca1e948fa25da3f3788ae3095026cedfb0d1f20dc538833f7b62d1da4a\": rpc error: code = NotFound desc = could not find container \"cd5853ca1e948fa25da3f3788ae3095026cedfb0d1f20dc538833f7b62d1da4a\": container with ID starting with cd5853ca1e948fa25da3f3788ae3095026cedfb0d1f20dc538833f7b62d1da4a not found: ID does not exist" Jul 06 19:57:09.260675 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.260640 2561 scope.go:117] "RemoveContainer" containerID="32a448a28e8c6b429436483a43730290030ac7a996744c82b59a79a81e354d56" Jul 06 19:57:09.260866 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:57:09.260850 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32a448a28e8c6b429436483a43730290030ac7a996744c82b59a79a81e354d56\": container with ID starting with 32a448a28e8c6b429436483a43730290030ac7a996744c82b59a79a81e354d56 not found: ID does not exist" containerID="32a448a28e8c6b429436483a43730290030ac7a996744c82b59a79a81e354d56" Jul 06 19:57:09.260905 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.260873 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32a448a28e8c6b429436483a43730290030ac7a996744c82b59a79a81e354d56"} err="failed to get container status \"32a448a28e8c6b429436483a43730290030ac7a996744c82b59a79a81e354d56\": rpc error: code = NotFound desc = could not find container \"32a448a28e8c6b429436483a43730290030ac7a996744c82b59a79a81e354d56\": container with ID starting with 32a448a28e8c6b429436483a43730290030ac7a996744c82b59a79a81e354d56 not found: ID does not exist" Jul 06 19:57:09.267613 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.267588 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6"] Jul 06 19:57:09.272325 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.272280 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7fd56226-openshift-default-67658b5bcf-72hp6"] Jul 06 19:57:09.687904 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:09.687870 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6261b27-2db7-466a-b40d-8a09805b31f5" path="/var/lib/kubelet/pods/b6261b27-2db7-466a-b40d-8a09805b31f5/volumes" Jul 06 19:57:23.946733 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:23.946697 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h"] Jul 06 19:57:23.947235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:23.947086 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6261b27-2db7-466a-b40d-8a09805b31f5" containerName="istio-proxy" Jul 06 19:57:23.947235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:23.947097 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6261b27-2db7-466a-b40d-8a09805b31f5" containerName="istio-proxy" Jul 06 19:57:23.947235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:23.947106 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6261b27-2db7-466a-b40d-8a09805b31f5" containerName="istio-proxy" Jul 06 19:57:23.947235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:23.947111 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6261b27-2db7-466a-b40d-8a09805b31f5" containerName="istio-proxy" Jul 06 19:57:23.947235 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:23.947180 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6261b27-2db7-466a-b40d-8a09805b31f5" containerName="istio-proxy" Jul 06 19:57:23.949271 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:23.949255 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:23.952935 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:23.952915 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f938018c-openshift-default-dockercfg-rvkvk\"" Jul 06 19:57:23.968569 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:23.968546 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h"] Jul 06 19:57:24.080568 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.080524 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86991217-90cc-41c1-a7da-7df81753cbd4-istiod-ca-cert\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.080750 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.080583 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86991217-90cc-41c1-a7da-7df81753cbd4-istio-token\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.080750 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.080632 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-istio-envoy\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.080750 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.080664 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-credential-socket\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.080750 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.080706 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq2pw\" (UniqueName: \"kubernetes.io/projected/86991217-90cc-41c1-a7da-7df81753cbd4-kube-api-access-tq2pw\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.080750 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.080727 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-workload-certs\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.080922 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.080768 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86991217-90cc-41c1-a7da-7df81753cbd4-istio-podinfo\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.080922 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.080824 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-istio-data\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.080922 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.080869 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-workload-socket\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.182367 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.182328 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq2pw\" (UniqueName: \"kubernetes.io/projected/86991217-90cc-41c1-a7da-7df81753cbd4-kube-api-access-tq2pw\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.182367 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.182369 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-workload-certs\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.182574 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.182395 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86991217-90cc-41c1-a7da-7df81753cbd4-istio-podinfo\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.182574 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.182540 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-istio-data\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.182650 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.182580 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-workload-socket\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.182751 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.182725 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86991217-90cc-41c1-a7da-7df81753cbd4-istiod-ca-cert\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.182823 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.182792 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-workload-certs\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.182880 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.182797 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86991217-90cc-41c1-a7da-7df81753cbd4-istio-token\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.182880 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.182834 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-workload-socket\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.182880 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.182862 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-istio-envoy\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.183028 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.182911 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-credential-socket\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.183028 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.182919 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-istio-data\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.183255 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.183231 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-credential-socket\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.183467 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.183448 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86991217-90cc-41c1-a7da-7df81753cbd4-istiod-ca-cert\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.184937 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.184912 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-istio-envoy\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.185022 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.184947 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86991217-90cc-41c1-a7da-7df81753cbd4-istio-podinfo\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.190614 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.190586 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86991217-90cc-41c1-a7da-7df81753cbd4-istio-token\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.190713 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.190698 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq2pw\" (UniqueName: \"kubernetes.io/projected/86991217-90cc-41c1-a7da-7df81753cbd4-kube-api-access-tq2pw\") pod \"e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.260696 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.260601 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:24.405562 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.405537 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h"] Jul 06 19:57:24.407066 ip-10-0-141-33 kubenswrapper[2561]: W0706 19:57:24.407036 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86991217_90cc_41c1_a7da_7df81753cbd4.slice/crio-ee85421d67d16cc9747854f91521a94750cb4cfddc147e78936094e62dea0ef6 WatchSource:0}: Error finding container ee85421d67d16cc9747854f91521a94750cb4cfddc147e78936094e62dea0ef6: Status 404 returned error can't find the container with id ee85421d67d16cc9747854f91521a94750cb4cfddc147e78936094e62dea0ef6 Jul 06 19:57:24.409313 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.409269 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:57:24.409393 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.409351 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:57:24.409393 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:24.409381 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:57:25.325042 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:25.325006 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" event={"ID":"86991217-90cc-41c1-a7da-7df81753cbd4","Type":"ContainerStarted","Data":"2ce8ccb51e9387f886c3ef2b57b6496dc4d443b670acfe4dbba56f4dcff35a59"} Jul 06 19:57:25.325448 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:25.325062 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" event={"ID":"86991217-90cc-41c1-a7da-7df81753cbd4","Type":"ContainerStarted","Data":"ee85421d67d16cc9747854f91521a94750cb4cfddc147e78936094e62dea0ef6"} Jul 06 19:57:25.352260 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:25.352213 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" podStartSLOduration=2.352197287 podStartE2EDuration="2.352197287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:57:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:57:25.348653647 +0000 UTC m=+1780.216329202" watchObservedRunningTime="2026-07-06 19:57:25.352197287 +0000 UTC m=+1780.219872842" Jul 06 19:57:26.261670 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:26.261640 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:26.266205 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:26.266181 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:26.329778 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:26.329750 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:26.330679 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:26.330658 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:38.400459 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:38.400426 2561 generic.go:356] "Generic (PLEG): container finished" podID="86991217-90cc-41c1-a7da-7df81753cbd4" containerID="2ce8ccb51e9387f886c3ef2b57b6496dc4d443b670acfe4dbba56f4dcff35a59" exitCode=0 Jul 06 19:57:38.400813 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:38.400479 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" event={"ID":"86991217-90cc-41c1-a7da-7df81753cbd4","Type":"ContainerDied","Data":"2ce8ccb51e9387f886c3ef2b57b6496dc4d443b670acfe4dbba56f4dcff35a59"} Jul 06 19:57:38.400876 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:38.400862 2561 scope.go:117] "RemoveContainer" containerID="2ce8ccb51e9387f886c3ef2b57b6496dc4d443b670acfe4dbba56f4dcff35a59" Jul 06 19:57:38.401858 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:38.401831 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:57:38.401906 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:38.401897 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:57:38.401941 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:38.401934 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 19:57:39.407609 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:39.407575 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" event={"ID":"86991217-90cc-41c1-a7da-7df81753cbd4","Type":"ContainerStarted","Data":"597f55de68f8ae1cd08ab76bf760a0a4b510af425681d19c947ed964210ac558"} Jul 06 19:57:40.261123 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:40.261083 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:40.262352 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:40.262328 2561 patch_prober.go:28] interesting pod/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 06 19:57:40.262466 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:40.262378 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" podUID="86991217-90cc-41c1-a7da-7df81753cbd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 06 19:57:41.261998 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:41.261960 2561 patch_prober.go:28] interesting pod/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 06 19:57:41.262418 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:41.262024 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" podUID="86991217-90cc-41c1-a7da-7df81753cbd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 06 19:57:41.329976 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:41.329945 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:42.261569 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:42.261531 2561 patch_prober.go:28] interesting pod/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 06 19:57:42.261752 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:42.261596 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" podUID="86991217-90cc-41c1-a7da-7df81753cbd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 06 19:57:43.264938 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:43.264910 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:57:43.265773 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:57:43.265755 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:59:01.976731 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:01.976695 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h"] Jul 06 19:59:01.977181 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:01.976967 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" podUID="86991217-90cc-41c1-a7da-7df81753cbd4" containerName="istio-proxy" containerID="cri-o://597f55de68f8ae1cd08ab76bf760a0a4b510af425681d19c947ed964210ac558" gracePeriod=30 Jul 06 19:59:07.235802 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.235772 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:59:07.312636 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.312560 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86991217-90cc-41c1-a7da-7df81753cbd4-istio-podinfo\") pod \"86991217-90cc-41c1-a7da-7df81753cbd4\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " Jul 06 19:59:07.312636 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.312613 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86991217-90cc-41c1-a7da-7df81753cbd4-istio-token\") pod \"86991217-90cc-41c1-a7da-7df81753cbd4\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " Jul 06 19:59:07.312834 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.312658 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-workload-socket\") pod \"86991217-90cc-41c1-a7da-7df81753cbd4\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " Jul 06 19:59:07.312834 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.312697 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-workload-certs\") pod \"86991217-90cc-41c1-a7da-7df81753cbd4\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " Jul 06 19:59:07.312834 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.312739 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tq2pw\" (UniqueName: \"kubernetes.io/projected/86991217-90cc-41c1-a7da-7df81753cbd4-kube-api-access-tq2pw\") pod \"86991217-90cc-41c1-a7da-7df81753cbd4\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " Jul 06 19:59:07.312834 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.312780 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-credential-socket\") pod \"86991217-90cc-41c1-a7da-7df81753cbd4\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " Jul 06 19:59:07.312834 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.312801 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-istio-data\") pod \"86991217-90cc-41c1-a7da-7df81753cbd4\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " Jul 06 19:59:07.313094 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.312842 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86991217-90cc-41c1-a7da-7df81753cbd4-istiod-ca-cert\") pod \"86991217-90cc-41c1-a7da-7df81753cbd4\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " Jul 06 19:59:07.313094 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.312887 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-istio-envoy\") pod \"86991217-90cc-41c1-a7da-7df81753cbd4\" (UID: \"86991217-90cc-41c1-a7da-7df81753cbd4\") " Jul 06 19:59:07.313094 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.313013 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "86991217-90cc-41c1-a7da-7df81753cbd4" (UID: "86991217-90cc-41c1-a7da-7df81753cbd4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:59:07.313359 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.313314 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "86991217-90cc-41c1-a7da-7df81753cbd4" (UID: "86991217-90cc-41c1-a7da-7df81753cbd4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:59:07.313359 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.313328 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "86991217-90cc-41c1-a7da-7df81753cbd4" (UID: "86991217-90cc-41c1-a7da-7df81753cbd4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:59:07.313538 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.313425 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-workload-certs\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:59:07.313771 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.313738 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86991217-90cc-41c1-a7da-7df81753cbd4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "86991217-90cc-41c1-a7da-7df81753cbd4" (UID: "86991217-90cc-41c1-a7da-7df81753cbd4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:59:07.314092 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.314061 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-istio-data" (OuterVolumeSpecName: "istio-data") pod "86991217-90cc-41c1-a7da-7df81753cbd4" (UID: "86991217-90cc-41c1-a7da-7df81753cbd4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:59:07.315204 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.315164 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86991217-90cc-41c1-a7da-7df81753cbd4-istio-token" (OuterVolumeSpecName: "istio-token") pod "86991217-90cc-41c1-a7da-7df81753cbd4" (UID: "86991217-90cc-41c1-a7da-7df81753cbd4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:59:07.315204 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.315163 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86991217-90cc-41c1-a7da-7df81753cbd4-kube-api-access-tq2pw" (OuterVolumeSpecName: "kube-api-access-tq2pw") pod "86991217-90cc-41c1-a7da-7df81753cbd4" (UID: "86991217-90cc-41c1-a7da-7df81753cbd4"). InnerVolumeSpecName "kube-api-access-tq2pw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:59:07.315408 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.315223 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "86991217-90cc-41c1-a7da-7df81753cbd4" (UID: "86991217-90cc-41c1-a7da-7df81753cbd4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:59:07.315497 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.315474 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/86991217-90cc-41c1-a7da-7df81753cbd4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "86991217-90cc-41c1-a7da-7df81753cbd4" (UID: "86991217-90cc-41c1-a7da-7df81753cbd4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:59:07.414160 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.414130 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86991217-90cc-41c1-a7da-7df81753cbd4-istio-podinfo\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:59:07.414160 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.414155 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86991217-90cc-41c1-a7da-7df81753cbd4-istio-token\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:59:07.414160 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.414166 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-workload-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:59:07.414401 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.414176 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tq2pw\" (UniqueName: \"kubernetes.io/projected/86991217-90cc-41c1-a7da-7df81753cbd4-kube-api-access-tq2pw\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:59:07.414401 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.414185 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-credential-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:59:07.414401 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.414193 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-istio-data\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:59:07.414401 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.414202 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86991217-90cc-41c1-a7da-7df81753cbd4-istiod-ca-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:59:07.414401 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.414209 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86991217-90cc-41c1-a7da-7df81753cbd4-istio-envoy\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 19:59:07.842371 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.842340 2561 generic.go:356] "Generic (PLEG): container finished" podID="86991217-90cc-41c1-a7da-7df81753cbd4" containerID="597f55de68f8ae1cd08ab76bf760a0a4b510af425681d19c947ed964210ac558" exitCode=0 Jul 06 19:59:07.842522 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.842387 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" Jul 06 19:59:07.842522 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.842428 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" event={"ID":"86991217-90cc-41c1-a7da-7df81753cbd4","Type":"ContainerDied","Data":"597f55de68f8ae1cd08ab76bf760a0a4b510af425681d19c947ed964210ac558"} Jul 06 19:59:07.842522 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.842481 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h" event={"ID":"86991217-90cc-41c1-a7da-7df81753cbd4","Type":"ContainerDied","Data":"ee85421d67d16cc9747854f91521a94750cb4cfddc147e78936094e62dea0ef6"} Jul 06 19:59:07.842522 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.842511 2561 scope.go:117] "RemoveContainer" containerID="597f55de68f8ae1cd08ab76bf760a0a4b510af425681d19c947ed964210ac558" Jul 06 19:59:07.850985 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.850968 2561 scope.go:117] "RemoveContainer" containerID="2ce8ccb51e9387f886c3ef2b57b6496dc4d443b670acfe4dbba56f4dcff35a59" Jul 06 19:59:07.858719 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.858702 2561 scope.go:117] "RemoveContainer" containerID="597f55de68f8ae1cd08ab76bf760a0a4b510af425681d19c947ed964210ac558" Jul 06 19:59:07.858973 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:59:07.858940 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"597f55de68f8ae1cd08ab76bf760a0a4b510af425681d19c947ed964210ac558\": container with ID starting with 597f55de68f8ae1cd08ab76bf760a0a4b510af425681d19c947ed964210ac558 not found: ID does not exist" containerID="597f55de68f8ae1cd08ab76bf760a0a4b510af425681d19c947ed964210ac558" Jul 06 19:59:07.859022 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.858982 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"597f55de68f8ae1cd08ab76bf760a0a4b510af425681d19c947ed964210ac558"} err="failed to get container status \"597f55de68f8ae1cd08ab76bf760a0a4b510af425681d19c947ed964210ac558\": rpc error: code = NotFound desc = could not find container \"597f55de68f8ae1cd08ab76bf760a0a4b510af425681d19c947ed964210ac558\": container with ID starting with 597f55de68f8ae1cd08ab76bf760a0a4b510af425681d19c947ed964210ac558 not found: ID does not exist" Jul 06 19:59:07.859022 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.859000 2561 scope.go:117] "RemoveContainer" containerID="2ce8ccb51e9387f886c3ef2b57b6496dc4d443b670acfe4dbba56f4dcff35a59" Jul 06 19:59:07.859261 ip-10-0-141-33 kubenswrapper[2561]: E0706 19:59:07.859244 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ce8ccb51e9387f886c3ef2b57b6496dc4d443b670acfe4dbba56f4dcff35a59\": container with ID starting with 2ce8ccb51e9387f886c3ef2b57b6496dc4d443b670acfe4dbba56f4dcff35a59 not found: ID does not exist" containerID="2ce8ccb51e9387f886c3ef2b57b6496dc4d443b670acfe4dbba56f4dcff35a59" Jul 06 19:59:07.859332 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.859267 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ce8ccb51e9387f886c3ef2b57b6496dc4d443b670acfe4dbba56f4dcff35a59"} err="failed to get container status \"2ce8ccb51e9387f886c3ef2b57b6496dc4d443b670acfe4dbba56f4dcff35a59\": rpc error: code = NotFound desc = could not find container \"2ce8ccb51e9387f886c3ef2b57b6496dc4d443b670acfe4dbba56f4dcff35a59\": container with ID starting with 2ce8ccb51e9387f886c3ef2b57b6496dc4d443b670acfe4dbba56f4dcff35a59 not found: ID does not exist" Jul 06 19:59:07.864449 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.864426 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h"] Jul 06 19:59:07.869797 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:07.869777 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f938018c-openshift-default-6db6c66947-mgx7h"] Jul 06 19:59:09.687521 ip-10-0-141-33 kubenswrapper[2561]: I0706 19:59:09.687487 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86991217-90cc-41c1-a7da-7df81753cbd4" path="/var/lib/kubelet/pods/86991217-90cc-41c1-a7da-7df81753cbd4/volumes" Jul 06 20:00:02.905762 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.905726 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k"] Jul 06 20:00:02.907245 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.907218 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86991217-90cc-41c1-a7da-7df81753cbd4" containerName="istio-proxy" Jul 06 20:00:02.907436 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.907423 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="86991217-90cc-41c1-a7da-7df81753cbd4" containerName="istio-proxy" Jul 06 20:00:02.907851 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.907831 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="86991217-90cc-41c1-a7da-7df81753cbd4" containerName="istio-proxy" Jul 06 20:00:02.907940 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.907856 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6261b27-2db7-466a-b40d-8a09805b31f5" containerName="istio-proxy" Jul 06 20:00:02.911499 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.911477 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:02.914158 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.914140 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-653e5c-openshift-default-dockercfg-bb92f\"" Jul 06 20:00:02.922334 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.922309 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k"] Jul 06 20:00:02.945364 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.945328 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-workload-socket\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:02.945527 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.945383 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-token\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:02.945527 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.945456 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-credential-socket\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:02.945527 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.945496 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-data\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:02.945527 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.945514 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwl4t\" (UniqueName: \"kubernetes.io/projected/ad528aa4-9d78-410c-b7ef-97f3552848c0-kube-api-access-gwl4t\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:02.945667 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.945537 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-podinfo\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:02.945667 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.945558 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad528aa4-9d78-410c-b7ef-97f3552848c0-istiod-ca-cert\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:02.945667 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.945606 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-envoy\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:02.945667 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:02.945640 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-workload-certs\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.046825 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.046783 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-token\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.047027 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.046846 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-credential-socket\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.047027 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.046878 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-data\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.047027 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.046900 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwl4t\" (UniqueName: \"kubernetes.io/projected/ad528aa4-9d78-410c-b7ef-97f3552848c0-kube-api-access-gwl4t\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.047027 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.046938 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-podinfo\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.047257 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.047081 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad528aa4-9d78-410c-b7ef-97f3552848c0-istiod-ca-cert\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.047257 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.047148 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-envoy\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.047257 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.047210 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-workload-certs\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.047459 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.047270 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-credential-socket\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.047459 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.047340 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-workload-socket\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.047459 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.047361 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-data\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.047631 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.047550 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-workload-certs\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.047703 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.047651 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-workload-socket\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.047872 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.047852 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad528aa4-9d78-410c-b7ef-97f3552848c0-istiod-ca-cert\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.049488 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.049463 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-envoy\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.050023 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.050000 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-podinfo\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.055009 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.054989 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-token\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.055359 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.055338 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwl4t\" (UniqueName: \"kubernetes.io/projected/ad528aa4-9d78-410c-b7ef-97f3552848c0-kube-api-access-gwl4t\") pod \"e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.224910 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.224875 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:03.363068 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.363042 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k"] Jul 06 20:00:03.364610 ip-10-0-141-33 kubenswrapper[2561]: W0706 20:00:03.364585 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad528aa4_9d78_410c_b7ef_97f3552848c0.slice/crio-0216806825fd399b09c8665fea5bfbbf927e92e7c1f46a42b17b664778c98d5b WatchSource:0}: Error finding container 0216806825fd399b09c8665fea5bfbbf927e92e7c1f46a42b17b664778c98d5b: Status 404 returned error can't find the container with id 0216806825fd399b09c8665fea5bfbbf927e92e7c1f46a42b17b664778c98d5b Jul 06 20:00:03.366844 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.366803 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:00:03.366928 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.366897 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:00:03.366974 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:03.366945 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:00:04.117268 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:04.117230 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" event={"ID":"ad528aa4-9d78-410c-b7ef-97f3552848c0","Type":"ContainerStarted","Data":"259cad84243fb9b3013c8d1314f92093dbdb0089a017ade7d1f32af76199ce44"} Jul 06 20:00:04.117741 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:04.117283 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" event={"ID":"ad528aa4-9d78-410c-b7ef-97f3552848c0","Type":"ContainerStarted","Data":"0216806825fd399b09c8665fea5bfbbf927e92e7c1f46a42b17b664778c98d5b"} Jul 06 20:00:04.144785 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:04.144733 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" podStartSLOduration=2.144715329 podStartE2EDuration="2.144715329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:00:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:00:04.144175668 +0000 UTC m=+1939.011851225" watchObservedRunningTime="2026-07-06 20:00:04.144715329 +0000 UTC m=+1939.012390884" Jul 06 20:00:04.225850 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:04.225809 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:04.230435 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:04.230411 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:05.122627 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:05.122597 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:05.123480 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:05.123461 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:12.042801 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.042767 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d"] Jul 06 20:00:12.043218 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.043204 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86991217-90cc-41c1-a7da-7df81753cbd4" containerName="istio-proxy" Jul 06 20:00:12.043259 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.043220 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="86991217-90cc-41c1-a7da-7df81753cbd4" containerName="istio-proxy" Jul 06 20:00:12.043316 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.043279 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="86991217-90cc-41c1-a7da-7df81753cbd4" containerName="istio-proxy" Jul 06 20:00:12.046500 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.046480 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.048842 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.048823 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-03ff21-openshift-default-dockercfg-2bfxb\"" Jul 06 20:00:12.061634 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.061610 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d"] Jul 06 20:00:12.137127 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.137093 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-podinfo\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.137421 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.137155 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-workload-certs\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.137421 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.137199 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istiod-ca-cert\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.137421 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.137225 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-data\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.137421 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.137330 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-credential-socket\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.137421 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.137384 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6srj\" (UniqueName: \"kubernetes.io/projected/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-kube-api-access-g6srj\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.137695 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.137506 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-envoy\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.137695 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.137543 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-workload-socket\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.137695 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.137580 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-token\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.238514 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.238477 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-token\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.238725 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.238540 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-podinfo\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.238725 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.238588 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-workload-certs\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.238725 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.238613 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istiod-ca-cert\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.238725 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.238641 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-data\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.238725 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.238672 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-credential-socket\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.238725 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.238704 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6srj\" (UniqueName: \"kubernetes.io/projected/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-kube-api-access-g6srj\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.239044 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.238801 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-envoy\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.239044 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.238838 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-workload-socket\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.239152 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.239058 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-credential-socket\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.239152 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.239118 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-data\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.239253 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.239186 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-workload-socket\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.239253 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.239197 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-workload-certs\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.239470 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.239448 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istiod-ca-cert\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.240979 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.240951 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-podinfo\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.240979 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.240975 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-envoy\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.254184 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.254160 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-token\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.254345 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.254265 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6srj\" (UniqueName: \"kubernetes.io/projected/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-kube-api-access-g6srj\") pod \"e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.356692 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.356609 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:12.500844 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.500814 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d"] Jul 06 20:00:12.502078 ip-10-0-141-33 kubenswrapper[2561]: W0706 20:00:12.502048 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33a73c6a_fdc0_463f_a0a5_fb5b8b61359b.slice/crio-d5b8c7f0a1dba4fa5c11c99b0f516a654a026f9608fc698c37c2e7f709d215c9 WatchSource:0}: Error finding container d5b8c7f0a1dba4fa5c11c99b0f516a654a026f9608fc698c37c2e7f709d215c9: Status 404 returned error can't find the container with id d5b8c7f0a1dba4fa5c11c99b0f516a654a026f9608fc698c37c2e7f709d215c9 Jul 06 20:00:12.504254 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.504221 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:00:12.504342 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.504302 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:00:12.504383 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:12.504352 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:00:13.167965 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:13.167929 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" event={"ID":"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b","Type":"ContainerStarted","Data":"4260652b928f11ab543ed30ab932bf9957e94c69f904e1b58681275d470e56f3"} Jul 06 20:00:13.168383 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:13.167987 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" event={"ID":"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b","Type":"ContainerStarted","Data":"d5b8c7f0a1dba4fa5c11c99b0f516a654a026f9608fc698c37c2e7f709d215c9"} Jul 06 20:00:13.190303 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:13.190253 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" podStartSLOduration=1.190238206 podStartE2EDuration="1.190238206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:00:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:00:13.187232564 +0000 UTC m=+1948.054908144" watchObservedRunningTime="2026-07-06 20:00:13.190238206 +0000 UTC m=+1948.057913762" Jul 06 20:00:13.357574 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:13.357543 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:13.362746 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:13.362718 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:14.172757 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:14.172724 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:14.173708 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:14.173689 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:20.124298 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:20.124260 2561 patch_prober.go:28] interesting pod/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 06 20:00:20.124708 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:20.124355 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 06 20:00:20.209501 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:20.209467 2561 generic.go:356] "Generic (PLEG): container finished" podID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerID="259cad84243fb9b3013c8d1314f92093dbdb0089a017ade7d1f32af76199ce44" exitCode=0 Jul 06 20:00:20.209687 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:20.209545 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" event={"ID":"ad528aa4-9d78-410c-b7ef-97f3552848c0","Type":"ContainerDied","Data":"259cad84243fb9b3013c8d1314f92093dbdb0089a017ade7d1f32af76199ce44"} Jul 06 20:00:20.210033 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:20.210016 2561 scope.go:117] "RemoveContainer" containerID="259cad84243fb9b3013c8d1314f92093dbdb0089a017ade7d1f32af76199ce44" Jul 06 20:00:20.211047 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:20.211023 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:00:20.211108 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:20.211088 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:00:20.211143 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:20.211124 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:00:21.216663 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:21.216625 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" event={"ID":"ad528aa4-9d78-410c-b7ef-97f3552848c0","Type":"ContainerStarted","Data":"726e9d6334ee5d7f9806b9b6efb8310ef87e8be0362aab5682f042f7bd0940ab"} Jul 06 20:00:21.225190 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:21.225159 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:21.225395 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:21.225373 2561 patch_prober.go:28] interesting pod/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 06 20:00:21.225475 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:21.225429 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 06 20:00:22.225819 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:22.225783 2561 patch_prober.go:28] interesting pod/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 06 20:00:22.226229 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:22.225843 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 06 20:00:23.225486 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:23.225449 2561 patch_prober.go:28] interesting pod/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 06 20:00:23.225748 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:23.225515 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 06 20:00:24.225715 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:24.225675 2561 patch_prober.go:28] interesting pod/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 06 20:00:24.226209 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:24.225737 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 06 20:00:25.225611 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:25.225577 2561 patch_prober.go:28] interesting pod/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 06 20:00:25.225789 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:25.225633 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 06 20:00:26.230714 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:26.230682 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:26.231164 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:26.231076 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:26.234889 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:26.234865 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:00:29.676942 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:29.675971 2561 patch_prober.go:28] interesting pod/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": read tcp 10.134.0.2:49656->10.134.0.43:15021: read: connection reset by peer" start-of-body= Jul 06 20:00:29.676942 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:29.676054 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" podUID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": read tcp 10.134.0.2:49656->10.134.0.43:15021: read: connection reset by peer" Jul 06 20:00:30.274019 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:30.273985 2561 generic.go:356] "Generic (PLEG): container finished" podID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerID="4260652b928f11ab543ed30ab932bf9957e94c69f904e1b58681275d470e56f3" exitCode=0 Jul 06 20:00:30.274192 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:30.274059 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" event={"ID":"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b","Type":"ContainerDied","Data":"4260652b928f11ab543ed30ab932bf9957e94c69f904e1b58681275d470e56f3"} Jul 06 20:00:30.274519 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:30.274504 2561 scope.go:117] "RemoveContainer" containerID="4260652b928f11ab543ed30ab932bf9957e94c69f904e1b58681275d470e56f3" Jul 06 20:00:30.275524 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:30.275493 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:00:30.275642 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:30.275574 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:00:30.275642 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:30.275622 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:00:31.280971 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:31.280927 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" event={"ID":"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b","Type":"ContainerStarted","Data":"acefe43048e5a7f80d50f8f0a221cfa1f56b0c151ecb7ff6d957c9343fb755e5"} Jul 06 20:00:31.357509 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:31.357470 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:31.357754 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:31.357726 2561 patch_prober.go:28] interesting pod/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 06 20:00:31.357828 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:31.357803 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" podUID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 06 20:00:32.357375 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:32.357345 2561 patch_prober.go:28] interesting pod/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 06 20:00:32.357759 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:32.357395 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" podUID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 06 20:00:33.357395 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:33.357359 2561 patch_prober.go:28] interesting pod/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 06 20:00:33.357789 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:33.357420 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" podUID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 06 20:00:34.357167 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:34.357124 2561 patch_prober.go:28] interesting pod/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 06 20:00:34.357414 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:34.357191 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" podUID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 06 20:00:35.361331 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:35.361283 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:35.361735 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:35.361575 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:00:35.362404 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:00:35.362381 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:02:08.179143 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:08.179066 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k"] Jul 06 20:02:08.179567 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:08.179274 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerName="istio-proxy" containerID="cri-o://726e9d6334ee5d7f9806b9b6efb8310ef87e8be0362aab5682f042f7bd0940ab" gracePeriod=30 Jul 06 20:02:11.233847 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:11.233813 2561 patch_prober.go:28] interesting pod/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 20:02:11.234215 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:11.233864 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 20:02:13.431867 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.431839 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:02:13.476408 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.476336 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-workload-socket\") pod \"ad528aa4-9d78-410c-b7ef-97f3552848c0\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " Jul 06 20:02:13.476408 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.476396 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-token\") pod \"ad528aa4-9d78-410c-b7ef-97f3552848c0\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " Jul 06 20:02:13.476612 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.476422 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-credential-socket\") pod \"ad528aa4-9d78-410c-b7ef-97f3552848c0\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " Jul 06 20:02:13.476612 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.476471 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwl4t\" (UniqueName: \"kubernetes.io/projected/ad528aa4-9d78-410c-b7ef-97f3552848c0-kube-api-access-gwl4t\") pod \"ad528aa4-9d78-410c-b7ef-97f3552848c0\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " Jul 06 20:02:13.476612 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.476508 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-workload-certs\") pod \"ad528aa4-9d78-410c-b7ef-97f3552848c0\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " Jul 06 20:02:13.476612 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.476582 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-envoy\") pod \"ad528aa4-9d78-410c-b7ef-97f3552848c0\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " Jul 06 20:02:13.476850 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.476642 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-data\") pod \"ad528aa4-9d78-410c-b7ef-97f3552848c0\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " Jul 06 20:02:13.476850 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.476641 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ad528aa4-9d78-410c-b7ef-97f3552848c0" (UID: "ad528aa4-9d78-410c-b7ef-97f3552848c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:13.476850 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.476668 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-podinfo\") pod \"ad528aa4-9d78-410c-b7ef-97f3552848c0\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " Jul 06 20:02:13.476850 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.476704 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad528aa4-9d78-410c-b7ef-97f3552848c0-istiod-ca-cert\") pod \"ad528aa4-9d78-410c-b7ef-97f3552848c0\" (UID: \"ad528aa4-9d78-410c-b7ef-97f3552848c0\") " Jul 06 20:02:13.476850 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.476781 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ad528aa4-9d78-410c-b7ef-97f3552848c0" (UID: "ad528aa4-9d78-410c-b7ef-97f3552848c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:13.476850 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.476827 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ad528aa4-9d78-410c-b7ef-97f3552848c0" (UID: "ad528aa4-9d78-410c-b7ef-97f3552848c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:13.477130 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.477096 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-workload-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:13.477130 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.477119 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-credential-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:13.477228 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.477133 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-workload-certs\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:13.477228 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.477098 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad528aa4-9d78-410c-b7ef-97f3552848c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ad528aa4-9d78-410c-b7ef-97f3552848c0" (UID: "ad528aa4-9d78-410c-b7ef-97f3552848c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:02:13.477572 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.477543 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "ad528aa4-9d78-410c-b7ef-97f3552848c0" (UID: "ad528aa4-9d78-410c-b7ef-97f3552848c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:13.478805 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.478781 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ad528aa4-9d78-410c-b7ef-97f3552848c0" (UID: "ad528aa4-9d78-410c-b7ef-97f3552848c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:13.478805 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.478790 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "ad528aa4-9d78-410c-b7ef-97f3552848c0" (UID: "ad528aa4-9d78-410c-b7ef-97f3552848c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:02:13.479205 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.479186 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ad528aa4-9d78-410c-b7ef-97f3552848c0" (UID: "ad528aa4-9d78-410c-b7ef-97f3552848c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:02:13.479266 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.479195 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad528aa4-9d78-410c-b7ef-97f3552848c0-kube-api-access-gwl4t" (OuterVolumeSpecName: "kube-api-access-gwl4t") pod "ad528aa4-9d78-410c-b7ef-97f3552848c0" (UID: "ad528aa4-9d78-410c-b7ef-97f3552848c0"). InnerVolumeSpecName "kube-api-access-gwl4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:02:13.578650 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.578614 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwl4t\" (UniqueName: \"kubernetes.io/projected/ad528aa4-9d78-410c-b7ef-97f3552848c0-kube-api-access-gwl4t\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:13.578650 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.578648 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-envoy\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:13.578650 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.578658 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-data\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:13.578885 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.578665 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-podinfo\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:13.578885 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.578673 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad528aa4-9d78-410c-b7ef-97f3552848c0-istiod-ca-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:13.578885 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.578681 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad528aa4-9d78-410c-b7ef-97f3552848c0-istio-token\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:13.795805 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.795714 2561 generic.go:356] "Generic (PLEG): container finished" podID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerID="726e9d6334ee5d7f9806b9b6efb8310ef87e8be0362aab5682f042f7bd0940ab" exitCode=0 Jul 06 20:02:13.795805 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.795763 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" Jul 06 20:02:13.795980 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.795802 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" event={"ID":"ad528aa4-9d78-410c-b7ef-97f3552848c0","Type":"ContainerDied","Data":"726e9d6334ee5d7f9806b9b6efb8310ef87e8be0362aab5682f042f7bd0940ab"} Jul 06 20:02:13.795980 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.795861 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k" event={"ID":"ad528aa4-9d78-410c-b7ef-97f3552848c0","Type":"ContainerDied","Data":"0216806825fd399b09c8665fea5bfbbf927e92e7c1f46a42b17b664778c98d5b"} Jul 06 20:02:13.795980 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.795890 2561 scope.go:117] "RemoveContainer" containerID="726e9d6334ee5d7f9806b9b6efb8310ef87e8be0362aab5682f042f7bd0940ab" Jul 06 20:02:13.804999 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.804980 2561 scope.go:117] "RemoveContainer" containerID="259cad84243fb9b3013c8d1314f92093dbdb0089a017ade7d1f32af76199ce44" Jul 06 20:02:13.814362 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.814343 2561 scope.go:117] "RemoveContainer" containerID="726e9d6334ee5d7f9806b9b6efb8310ef87e8be0362aab5682f042f7bd0940ab" Jul 06 20:02:13.814645 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.814621 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k"] Jul 06 20:02:13.814749 ip-10-0-141-33 kubenswrapper[2561]: E0706 20:02:13.814647 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"726e9d6334ee5d7f9806b9b6efb8310ef87e8be0362aab5682f042f7bd0940ab\": container with ID starting with 726e9d6334ee5d7f9806b9b6efb8310ef87e8be0362aab5682f042f7bd0940ab not found: ID does not exist" containerID="726e9d6334ee5d7f9806b9b6efb8310ef87e8be0362aab5682f042f7bd0940ab" Jul 06 20:02:13.814749 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.814672 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"726e9d6334ee5d7f9806b9b6efb8310ef87e8be0362aab5682f042f7bd0940ab"} err="failed to get container status \"726e9d6334ee5d7f9806b9b6efb8310ef87e8be0362aab5682f042f7bd0940ab\": rpc error: code = NotFound desc = could not find container \"726e9d6334ee5d7f9806b9b6efb8310ef87e8be0362aab5682f042f7bd0940ab\": container with ID starting with 726e9d6334ee5d7f9806b9b6efb8310ef87e8be0362aab5682f042f7bd0940ab not found: ID does not exist" Jul 06 20:02:13.814749 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.814689 2561 scope.go:117] "RemoveContainer" containerID="259cad84243fb9b3013c8d1314f92093dbdb0089a017ade7d1f32af76199ce44" Jul 06 20:02:13.814937 ip-10-0-141-33 kubenswrapper[2561]: E0706 20:02:13.814922 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"259cad84243fb9b3013c8d1314f92093dbdb0089a017ade7d1f32af76199ce44\": container with ID starting with 259cad84243fb9b3013c8d1314f92093dbdb0089a017ade7d1f32af76199ce44 not found: ID does not exist" containerID="259cad84243fb9b3013c8d1314f92093dbdb0089a017ade7d1f32af76199ce44" Jul 06 20:02:13.814979 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.814943 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"259cad84243fb9b3013c8d1314f92093dbdb0089a017ade7d1f32af76199ce44"} err="failed to get container status \"259cad84243fb9b3013c8d1314f92093dbdb0089a017ade7d1f32af76199ce44\": rpc error: code = NotFound desc = could not find container \"259cad84243fb9b3013c8d1314f92093dbdb0089a017ade7d1f32af76199ce44\": container with ID starting with 259cad84243fb9b3013c8d1314f92093dbdb0089a017ade7d1f32af76199ce44 not found: ID does not exist" Jul 06 20:02:13.821974 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:13.821952 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-653e5c-openshift-default-67dbc99d7-k254k"] Jul 06 20:02:15.213113 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:15.213082 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d"] Jul 06 20:02:15.213544 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:15.213317 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" podUID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerName="istio-proxy" containerID="cri-o://acefe43048e5a7f80d50f8f0a221cfa1f56b0c151ecb7ff6d957c9343fb755e5" gracePeriod=30 Jul 06 20:02:15.688614 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:15.688587 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" path="/var/lib/kubelet/pods/ad528aa4-9d78-410c-b7ef-97f3552848c0/volumes" Jul 06 20:02:16.205816 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.205785 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9"] Jul 06 20:02:16.206231 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.206217 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerName="istio-proxy" Jul 06 20:02:16.206282 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.206232 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerName="istio-proxy" Jul 06 20:02:16.206282 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.206242 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerName="istio-proxy" Jul 06 20:02:16.206282 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.206247 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerName="istio-proxy" Jul 06 20:02:16.206404 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.206340 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerName="istio-proxy" Jul 06 20:02:16.206404 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.206349 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad528aa4-9d78-410c-b7ef-97f3552848c0" containerName="istio-proxy" Jul 06 20:02:16.211691 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.211671 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.214324 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.214279 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f1eed6-openshift-default-dockercfg-kwzzl\"" Jul 06 20:02:16.223549 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.223529 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9"] Jul 06 20:02:16.307812 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.307778 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-token\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.307999 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.307843 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-workload-certs\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.307999 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.307890 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92422236-ab8d-44b4-8d7b-263b8490dca7-istiod-ca-cert\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.308090 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.308011 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-credential-socket\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.308139 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.308088 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-data\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.308139 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.308123 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-workload-socket\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.308220 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.308166 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-podinfo\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.308220 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.308199 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtlqb\" (UniqueName: \"kubernetes.io/projected/92422236-ab8d-44b4-8d7b-263b8490dca7-kube-api-access-xtlqb\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.308327 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.308230 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-envoy\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.409179 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.409141 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-podinfo\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.409179 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.409181 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtlqb\" (UniqueName: \"kubernetes.io/projected/92422236-ab8d-44b4-8d7b-263b8490dca7-kube-api-access-xtlqb\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.409460 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.409201 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-envoy\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.409460 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.409243 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-token\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.409460 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.409267 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-workload-certs\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.409460 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.409314 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92422236-ab8d-44b4-8d7b-263b8490dca7-istiod-ca-cert\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.409460 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.409366 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-credential-socket\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.409460 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.409402 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-data\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.409460 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.409423 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-workload-socket\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.409906 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.409722 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-workload-certs\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.409906 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.409797 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-credential-socket\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.409906 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.409851 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-data\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.409906 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.409868 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-workload-socket\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.410060 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.410043 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92422236-ab8d-44b4-8d7b-263b8490dca7-istiod-ca-cert\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.411548 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.411521 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-envoy\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.411837 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.411820 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-podinfo\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.418041 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.418014 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtlqb\" (UniqueName: \"kubernetes.io/projected/92422236-ab8d-44b4-8d7b-263b8490dca7-kube-api-access-xtlqb\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.418585 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.418565 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-token\") pod \"e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.522916 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.522835 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:16.662751 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.662723 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9"] Jul 06 20:02:16.664489 ip-10-0-141-33 kubenswrapper[2561]: W0706 20:02:16.664459 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92422236_ab8d_44b4_8d7b_263b8490dca7.slice/crio-b14740c2d9786a78c21b4a7ee3a8b0fc798bf4b9e929b4f776ba65a747cec82a WatchSource:0}: Error finding container b14740c2d9786a78c21b4a7ee3a8b0fc798bf4b9e929b4f776ba65a747cec82a: Status 404 returned error can't find the container with id b14740c2d9786a78c21b4a7ee3a8b0fc798bf4b9e929b4f776ba65a747cec82a Jul 06 20:02:16.666757 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.666726 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:16.666848 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.666793 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:16.666848 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.666820 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:16.823412 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.823274 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" event={"ID":"92422236-ab8d-44b4-8d7b-263b8490dca7","Type":"ContainerStarted","Data":"319fc5d3cb374f87358981990f49c7ed09893c982b27c0a61f8fd49689fea9d4"} Jul 06 20:02:16.823412 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.823396 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" event={"ID":"92422236-ab8d-44b4-8d7b-263b8490dca7","Type":"ContainerStarted","Data":"b14740c2d9786a78c21b4a7ee3a8b0fc798bf4b9e929b4f776ba65a747cec82a"} Jul 06 20:02:16.847054 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:16.847007 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" podStartSLOduration=0.846991922 podStartE2EDuration="846.991922ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:02:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:02:16.84550324 +0000 UTC m=+2071.713178818" watchObservedRunningTime="2026-07-06 20:02:16.846991922 +0000 UTC m=+2071.714667478" Jul 06 20:02:17.523432 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:17.523392 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:17.528458 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:17.528432 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:17.827945 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:17.827863 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:17.828899 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:17.828881 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:20.362902 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.362871 2561 patch_prober.go:28] interesting pod/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 06 20:02:20.363253 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.362948 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" podUID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 06 20:02:20.465931 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.465907 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:02:20.551740 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.551647 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istiod-ca-cert\") pod \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " Jul 06 20:02:20.551740 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.551705 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6srj\" (UniqueName: \"kubernetes.io/projected/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-kube-api-access-g6srj\") pod \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " Jul 06 20:02:20.551946 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.551832 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-data\") pod \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " Jul 06 20:02:20.551946 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.551866 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-credential-socket\") pod \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " Jul 06 20:02:20.551946 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.551893 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-token\") pod \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " Jul 06 20:02:20.551946 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.551926 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-podinfo\") pod \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " Jul 06 20:02:20.552152 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.551951 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-workload-certs\") pod \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " Jul 06 20:02:20.552152 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.552013 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-workload-socket\") pod \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " Jul 06 20:02:20.552152 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.552040 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-envoy\") pod \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\" (UID: \"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b\") " Jul 06 20:02:20.552152 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.552059 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" (UID: "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:02:20.552408 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.552187 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" (UID: "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:20.552461 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.552397 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" (UID: "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:20.552461 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.552424 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" (UID: "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:20.552564 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.552527 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-credential-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:20.552564 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.552543 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-workload-certs\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:20.552564 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.552557 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-workload-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:20.552717 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.552568 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istiod-ca-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:20.552847 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.552829 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-data" (OuterVolumeSpecName: "istio-data") pod "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" (UID: "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:20.554165 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.554140 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" (UID: "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:02:20.554282 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.554163 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-kube-api-access-g6srj" (OuterVolumeSpecName: "kube-api-access-g6srj") pod "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" (UID: "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b"). InnerVolumeSpecName "kube-api-access-g6srj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:02:20.554455 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.554434 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-token" (OuterVolumeSpecName: "istio-token") pod "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" (UID: "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:02:20.554533 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.554483 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" (UID: "33a73c6a-fdc0-463f-a0a5-fb5b8b61359b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:20.653752 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.653718 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-envoy\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:20.653752 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.653746 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6srj\" (UniqueName: \"kubernetes.io/projected/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-kube-api-access-g6srj\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:20.653752 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.653755 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-data\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:20.653967 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.653768 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-token\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:20.653967 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.653775 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b-istio-podinfo\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:20.843952 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.843858 2561 generic.go:356] "Generic (PLEG): container finished" podID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerID="acefe43048e5a7f80d50f8f0a221cfa1f56b0c151ecb7ff6d957c9343fb755e5" exitCode=0 Jul 06 20:02:20.843952 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.843906 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" Jul 06 20:02:20.843952 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.843938 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" event={"ID":"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b","Type":"ContainerDied","Data":"acefe43048e5a7f80d50f8f0a221cfa1f56b0c151ecb7ff6d957c9343fb755e5"} Jul 06 20:02:20.844167 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.843991 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d" event={"ID":"33a73c6a-fdc0-463f-a0a5-fb5b8b61359b","Type":"ContainerDied","Data":"d5b8c7f0a1dba4fa5c11c99b0f516a654a026f9608fc698c37c2e7f709d215c9"} Jul 06 20:02:20.844167 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.844013 2561 scope.go:117] "RemoveContainer" containerID="acefe43048e5a7f80d50f8f0a221cfa1f56b0c151ecb7ff6d957c9343fb755e5" Jul 06 20:02:20.853083 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.853067 2561 scope.go:117] "RemoveContainer" containerID="4260652b928f11ab543ed30ab932bf9957e94c69f904e1b58681275d470e56f3" Jul 06 20:02:20.860925 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.860910 2561 scope.go:117] "RemoveContainer" containerID="acefe43048e5a7f80d50f8f0a221cfa1f56b0c151ecb7ff6d957c9343fb755e5" Jul 06 20:02:20.861148 ip-10-0-141-33 kubenswrapper[2561]: E0706 20:02:20.861133 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acefe43048e5a7f80d50f8f0a221cfa1f56b0c151ecb7ff6d957c9343fb755e5\": container with ID starting with acefe43048e5a7f80d50f8f0a221cfa1f56b0c151ecb7ff6d957c9343fb755e5 not found: ID does not exist" containerID="acefe43048e5a7f80d50f8f0a221cfa1f56b0c151ecb7ff6d957c9343fb755e5" Jul 06 20:02:20.861191 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.861156 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acefe43048e5a7f80d50f8f0a221cfa1f56b0c151ecb7ff6d957c9343fb755e5"} err="failed to get container status \"acefe43048e5a7f80d50f8f0a221cfa1f56b0c151ecb7ff6d957c9343fb755e5\": rpc error: code = NotFound desc = could not find container \"acefe43048e5a7f80d50f8f0a221cfa1f56b0c151ecb7ff6d957c9343fb755e5\": container with ID starting with acefe43048e5a7f80d50f8f0a221cfa1f56b0c151ecb7ff6d957c9343fb755e5 not found: ID does not exist" Jul 06 20:02:20.861191 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.861172 2561 scope.go:117] "RemoveContainer" containerID="4260652b928f11ab543ed30ab932bf9957e94c69f904e1b58681275d470e56f3" Jul 06 20:02:20.861391 ip-10-0-141-33 kubenswrapper[2561]: E0706 20:02:20.861373 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4260652b928f11ab543ed30ab932bf9957e94c69f904e1b58681275d470e56f3\": container with ID starting with 4260652b928f11ab543ed30ab932bf9957e94c69f904e1b58681275d470e56f3 not found: ID does not exist" containerID="4260652b928f11ab543ed30ab932bf9957e94c69f904e1b58681275d470e56f3" Jul 06 20:02:20.861441 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.861397 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4260652b928f11ab543ed30ab932bf9957e94c69f904e1b58681275d470e56f3"} err="failed to get container status \"4260652b928f11ab543ed30ab932bf9957e94c69f904e1b58681275d470e56f3\": rpc error: code = NotFound desc = could not find container \"4260652b928f11ab543ed30ab932bf9957e94c69f904e1b58681275d470e56f3\": container with ID starting with 4260652b928f11ab543ed30ab932bf9957e94c69f904e1b58681275d470e56f3 not found: ID does not exist" Jul 06 20:02:20.868383 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.868362 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d"] Jul 06 20:02:20.873115 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:20.873095 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-03ff21-openshift-default-857569bcff-tsz6d"] Jul 06 20:02:21.687062 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:21.687030 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" path="/var/lib/kubelet/pods/33a73c6a-fdc0-463f-a0a5-fb5b8b61359b/volumes" Jul 06 20:02:24.214230 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.214189 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf"] Jul 06 20:02:24.214857 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.214835 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerName="istio-proxy" Jul 06 20:02:24.214937 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.214861 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerName="istio-proxy" Jul 06 20:02:24.214937 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.214894 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerName="istio-proxy" Jul 06 20:02:24.214937 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.214904 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerName="istio-proxy" Jul 06 20:02:24.215085 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.215023 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerName="istio-proxy" Jul 06 20:02:24.215085 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.215034 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="33a73c6a-fdc0-463f-a0a5-fb5b8b61359b" containerName="istio-proxy" Jul 06 20:02:24.219089 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.219069 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.222073 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.222052 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-0e21dc-openshift-default-dockercfg-msbv6\"" Jul 06 20:02:24.232684 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.232659 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf"] Jul 06 20:02:24.288697 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.288658 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-token\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.288872 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.288732 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-credential-socket\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.288872 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.288774 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istiod-ca-cert\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.288872 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.288802 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-workload-socket\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.288872 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.288824 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-envoy\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.288872 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.288847 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-workload-certs\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.288872 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.288868 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-podinfo\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.289116 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.288910 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-data\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.289116 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.288963 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nbfj\" (UniqueName: \"kubernetes.io/projected/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-kube-api-access-7nbfj\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.389920 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.389873 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-token\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.390152 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.389965 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-credential-socket\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.390152 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.390004 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istiod-ca-cert\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.390152 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.390032 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-workload-socket\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.390152 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.390052 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-envoy\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.390152 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.390089 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-workload-certs\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.390152 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.390127 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-podinfo\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.390152 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.390153 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-data\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.390562 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.390208 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nbfj\" (UniqueName: \"kubernetes.io/projected/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-kube-api-access-7nbfj\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.390562 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.390490 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-credential-socket\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.390673 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.390626 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-data\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.390733 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.390716 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-workload-socket\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.390787 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.390760 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-workload-certs\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.390961 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.390941 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istiod-ca-cert\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.392409 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.392387 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-envoy\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.392728 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.392705 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-podinfo\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.402024 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.402000 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-token\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.402199 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.402178 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nbfj\" (UniqueName: \"kubernetes.io/projected/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-kube-api-access-7nbfj\") pod \"e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.530254 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.530164 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:24.673284 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.673255 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf"] Jul 06 20:02:24.674754 ip-10-0-141-33 kubenswrapper[2561]: W0706 20:02:24.674728 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44e8ab0b_1a75_4d0e_9c9c_3ba9dfd3feb6.slice/crio-7f5fc21cf77ba9dba3b8c16e2674cba6c010db0c62e5ec14375e8034e1760828 WatchSource:0}: Error finding container 7f5fc21cf77ba9dba3b8c16e2674cba6c010db0c62e5ec14375e8034e1760828: Status 404 returned error can't find the container with id 7f5fc21cf77ba9dba3b8c16e2674cba6c010db0c62e5ec14375e8034e1760828 Jul 06 20:02:24.676871 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.676839 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:24.676953 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.676899 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:24.676953 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.676926 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:24.870280 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.870173 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" event={"ID":"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6","Type":"ContainerStarted","Data":"0e4de81e12952e83e23fc2052c5ceb96329a233c78ee8f8d90a1e57d0b810afa"} Jul 06 20:02:24.870280 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.870234 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" event={"ID":"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6","Type":"ContainerStarted","Data":"7f5fc21cf77ba9dba3b8c16e2674cba6c010db0c62e5ec14375e8034e1760828"} Jul 06 20:02:24.896561 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:24.896499 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" podStartSLOduration=0.896476733 podStartE2EDuration="896.476733ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:02:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:02:24.893315066 +0000 UTC m=+2079.760990620" watchObservedRunningTime="2026-07-06 20:02:24.896476733 +0000 UTC m=+2079.764152294" Jul 06 20:02:25.531343 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:25.531314 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:25.536195 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:25.536174 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:25.876343 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:25.876253 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:25.877210 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:25.877193 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:33.516660 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:33.515567 2561 patch_prober.go:28] interesting pod/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": read tcp 10.134.0.2:33806->10.134.0.44:15021: read: connection reset by peer" start-of-body= Jul 06 20:02:33.516660 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:33.515638 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" podUID="92422236-ab8d-44b4-8d7b-263b8490dca7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": read tcp 10.134.0.2:33806->10.134.0.44:15021: read: connection reset by peer" Jul 06 20:02:33.917423 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:33.917392 2561 generic.go:356] "Generic (PLEG): container finished" podID="92422236-ab8d-44b4-8d7b-263b8490dca7" containerID="319fc5d3cb374f87358981990f49c7ed09893c982b27c0a61f8fd49689fea9d4" exitCode=0 Jul 06 20:02:33.917591 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:33.917472 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" event={"ID":"92422236-ab8d-44b4-8d7b-263b8490dca7","Type":"ContainerDied","Data":"319fc5d3cb374f87358981990f49c7ed09893c982b27c0a61f8fd49689fea9d4"} Jul 06 20:02:33.917930 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:33.917915 2561 scope.go:117] "RemoveContainer" containerID="319fc5d3cb374f87358981990f49c7ed09893c982b27c0a61f8fd49689fea9d4" Jul 06 20:02:33.918955 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:33.918927 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:33.919012 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:33.918996 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:33.919047 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:33.919030 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:34.923743 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:34.923711 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" event={"ID":"92422236-ab8d-44b4-8d7b-263b8490dca7","Type":"ContainerStarted","Data":"84afc32ae0f8bf5c29b2c8dc5310ddf9743e288b8c9adbfa795817d00e38d57c"} Jul 06 20:02:35.523572 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:35.523541 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:35.523830 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:35.523737 2561 patch_prober.go:28] interesting pod/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 06 20:02:35.523960 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:35.523941 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" podUID="92422236-ab8d-44b4-8d7b-263b8490dca7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 06 20:02:36.523581 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:36.523541 2561 patch_prober.go:28] interesting pod/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 06 20:02:36.523997 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:36.523601 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" podUID="92422236-ab8d-44b4-8d7b-263b8490dca7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 06 20:02:37.523824 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:37.523782 2561 patch_prober.go:28] interesting pod/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 06 20:02:37.524320 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:37.523842 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" podUID="92422236-ab8d-44b4-8d7b-263b8490dca7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 06 20:02:38.523688 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:38.523643 2561 patch_prober.go:28] interesting pod/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 06 20:02:38.523872 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:38.523702 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" podUID="92422236-ab8d-44b4-8d7b-263b8490dca7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 06 20:02:39.527831 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:39.527796 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:39.528346 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:39.528324 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:39.528960 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:39.528937 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:41.200729 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:41.199716 2561 patch_prober.go:28] interesting pod/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:54842->10.134.0.45:15021: read: connection reset by peer" start-of-body= Jul 06 20:02:41.200729 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:41.199779 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" podUID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:54842->10.134.0.45:15021: read: connection reset by peer" Jul 06 20:02:41.970011 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:41.969979 2561 generic.go:356] "Generic (PLEG): container finished" podID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" containerID="0e4de81e12952e83e23fc2052c5ceb96329a233c78ee8f8d90a1e57d0b810afa" exitCode=0 Jul 06 20:02:41.970183 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:41.970051 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" event={"ID":"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6","Type":"ContainerDied","Data":"0e4de81e12952e83e23fc2052c5ceb96329a233c78ee8f8d90a1e57d0b810afa"} Jul 06 20:02:41.970492 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:41.970477 2561 scope.go:117] "RemoveContainer" containerID="0e4de81e12952e83e23fc2052c5ceb96329a233c78ee8f8d90a1e57d0b810afa" Jul 06 20:02:41.971429 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:41.971406 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:41.971500 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:41.971486 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:41.971538 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:41.971527 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:42.977274 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:42.977238 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" event={"ID":"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6","Type":"ContainerStarted","Data":"5e508e5e980ba216cc739faf5743ad08642642736ceb93abd52fcd12f18f3e6d"} Jul 06 20:02:43.531384 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:43.531349 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:43.531748 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:43.531664 2561 patch_prober.go:28] interesting pod/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 06 20:02:43.531922 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:43.531896 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" podUID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 06 20:02:44.530793 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:44.530747 2561 patch_prober.go:28] interesting pod/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 06 20:02:44.531308 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:44.530817 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" podUID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 06 20:02:45.532894 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:45.532847 2561 patch_prober.go:28] interesting pod/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 06 20:02:45.533385 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:45.532917 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" podUID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 06 20:02:46.531634 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:46.531588 2561 patch_prober.go:28] interesting pod/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 06 20:02:46.531840 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:46.531656 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" podUID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 06 20:02:47.287379 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:47.287347 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9"] Jul 06 20:02:47.287768 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:47.287543 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" podUID="92422236-ab8d-44b4-8d7b-263b8490dca7" containerName="istio-proxy" containerID="cri-o://84afc32ae0f8bf5c29b2c8dc5310ddf9743e288b8c9adbfa795817d00e38d57c" gracePeriod=30 Jul 06 20:02:47.534184 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:47.534153 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:47.534531 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:47.534510 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:47.535251 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:47.535234 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:02:52.535091 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.535066 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:52.600989 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.600896 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-envoy\") pod \"92422236-ab8d-44b4-8d7b-263b8490dca7\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " Jul 06 20:02:52.600989 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.600938 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-workload-socket\") pod \"92422236-ab8d-44b4-8d7b-263b8490dca7\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " Jul 06 20:02:52.600989 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.600970 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtlqb\" (UniqueName: \"kubernetes.io/projected/92422236-ab8d-44b4-8d7b-263b8490dca7-kube-api-access-xtlqb\") pod \"92422236-ab8d-44b4-8d7b-263b8490dca7\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " Jul 06 20:02:52.601259 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.600993 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-data\") pod \"92422236-ab8d-44b4-8d7b-263b8490dca7\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " Jul 06 20:02:52.601259 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.601038 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-credential-socket\") pod \"92422236-ab8d-44b4-8d7b-263b8490dca7\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " Jul 06 20:02:52.601259 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.601055 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-podinfo\") pod \"92422236-ab8d-44b4-8d7b-263b8490dca7\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " Jul 06 20:02:52.601259 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.601098 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92422236-ab8d-44b4-8d7b-263b8490dca7-istiod-ca-cert\") pod \"92422236-ab8d-44b4-8d7b-263b8490dca7\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " Jul 06 20:02:52.601259 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.601179 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-token\") pod \"92422236-ab8d-44b4-8d7b-263b8490dca7\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " Jul 06 20:02:52.601259 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.601211 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-workload-certs\") pod \"92422236-ab8d-44b4-8d7b-263b8490dca7\" (UID: \"92422236-ab8d-44b4-8d7b-263b8490dca7\") " Jul 06 20:02:52.601259 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.601206 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "92422236-ab8d-44b4-8d7b-263b8490dca7" (UID: "92422236-ab8d-44b4-8d7b-263b8490dca7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:52.601643 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.601368 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "92422236-ab8d-44b4-8d7b-263b8490dca7" (UID: "92422236-ab8d-44b4-8d7b-263b8490dca7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:52.601643 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.601592 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "92422236-ab8d-44b4-8d7b-263b8490dca7" (UID: "92422236-ab8d-44b4-8d7b-263b8490dca7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:52.601643 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.601618 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92422236-ab8d-44b4-8d7b-263b8490dca7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "92422236-ab8d-44b4-8d7b-263b8490dca7" (UID: "92422236-ab8d-44b4-8d7b-263b8490dca7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:02:52.601643 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.601631 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-credential-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:52.601854 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.601652 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-workload-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:52.602045 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.602023 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-data" (OuterVolumeSpecName: "istio-data") pod "92422236-ab8d-44b4-8d7b-263b8490dca7" (UID: "92422236-ab8d-44b4-8d7b-263b8490dca7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:52.603365 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.603332 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "92422236-ab8d-44b4-8d7b-263b8490dca7" (UID: "92422236-ab8d-44b4-8d7b-263b8490dca7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:02:52.603489 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.603379 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-token" (OuterVolumeSpecName: "istio-token") pod "92422236-ab8d-44b4-8d7b-263b8490dca7" (UID: "92422236-ab8d-44b4-8d7b-263b8490dca7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:02:52.603489 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.603456 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "92422236-ab8d-44b4-8d7b-263b8490dca7" (UID: "92422236-ab8d-44b4-8d7b-263b8490dca7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:52.603571 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.603485 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92422236-ab8d-44b4-8d7b-263b8490dca7-kube-api-access-xtlqb" (OuterVolumeSpecName: "kube-api-access-xtlqb") pod "92422236-ab8d-44b4-8d7b-263b8490dca7" (UID: "92422236-ab8d-44b4-8d7b-263b8490dca7"). InnerVolumeSpecName "kube-api-access-xtlqb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:02:52.702363 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.702320 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-podinfo\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:52.702363 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.702352 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92422236-ab8d-44b4-8d7b-263b8490dca7-istiod-ca-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:52.702363 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.702360 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-token\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:52.702363 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.702368 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-workload-certs\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:52.702363 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.702375 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-envoy\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:52.702671 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.702383 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtlqb\" (UniqueName: \"kubernetes.io/projected/92422236-ab8d-44b4-8d7b-263b8490dca7-kube-api-access-xtlqb\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:52.702671 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:52.702392 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92422236-ab8d-44b4-8d7b-263b8490dca7-istio-data\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:02:53.028778 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:53.028747 2561 generic.go:356] "Generic (PLEG): container finished" podID="92422236-ab8d-44b4-8d7b-263b8490dca7" containerID="84afc32ae0f8bf5c29b2c8dc5310ddf9743e288b8c9adbfa795817d00e38d57c" exitCode=0 Jul 06 20:02:53.028987 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:53.028795 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" Jul 06 20:02:53.028987 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:53.028832 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" event={"ID":"92422236-ab8d-44b4-8d7b-263b8490dca7","Type":"ContainerDied","Data":"84afc32ae0f8bf5c29b2c8dc5310ddf9743e288b8c9adbfa795817d00e38d57c"} Jul 06 20:02:53.028987 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:53.028893 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9" event={"ID":"92422236-ab8d-44b4-8d7b-263b8490dca7","Type":"ContainerDied","Data":"b14740c2d9786a78c21b4a7ee3a8b0fc798bf4b9e929b4f776ba65a747cec82a"} Jul 06 20:02:53.028987 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:53.028920 2561 scope.go:117] "RemoveContainer" containerID="84afc32ae0f8bf5c29b2c8dc5310ddf9743e288b8c9adbfa795817d00e38d57c" Jul 06 20:02:53.037845 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:53.037827 2561 scope.go:117] "RemoveContainer" containerID="319fc5d3cb374f87358981990f49c7ed09893c982b27c0a61f8fd49689fea9d4" Jul 06 20:02:53.045223 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:53.045207 2561 scope.go:117] "RemoveContainer" containerID="84afc32ae0f8bf5c29b2c8dc5310ddf9743e288b8c9adbfa795817d00e38d57c" Jul 06 20:02:53.045491 ip-10-0-141-33 kubenswrapper[2561]: E0706 20:02:53.045472 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84afc32ae0f8bf5c29b2c8dc5310ddf9743e288b8c9adbfa795817d00e38d57c\": container with ID starting with 84afc32ae0f8bf5c29b2c8dc5310ddf9743e288b8c9adbfa795817d00e38d57c not found: ID does not exist" containerID="84afc32ae0f8bf5c29b2c8dc5310ddf9743e288b8c9adbfa795817d00e38d57c" Jul 06 20:02:53.045538 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:53.045501 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84afc32ae0f8bf5c29b2c8dc5310ddf9743e288b8c9adbfa795817d00e38d57c"} err="failed to get container status \"84afc32ae0f8bf5c29b2c8dc5310ddf9743e288b8c9adbfa795817d00e38d57c\": rpc error: code = NotFound desc = could not find container \"84afc32ae0f8bf5c29b2c8dc5310ddf9743e288b8c9adbfa795817d00e38d57c\": container with ID starting with 84afc32ae0f8bf5c29b2c8dc5310ddf9743e288b8c9adbfa795817d00e38d57c not found: ID does not exist" Jul 06 20:02:53.045538 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:53.045518 2561 scope.go:117] "RemoveContainer" containerID="319fc5d3cb374f87358981990f49c7ed09893c982b27c0a61f8fd49689fea9d4" Jul 06 20:02:53.045714 ip-10-0-141-33 kubenswrapper[2561]: E0706 20:02:53.045700 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"319fc5d3cb374f87358981990f49c7ed09893c982b27c0a61f8fd49689fea9d4\": container with ID starting with 319fc5d3cb374f87358981990f49c7ed09893c982b27c0a61f8fd49689fea9d4 not found: ID does not exist" containerID="319fc5d3cb374f87358981990f49c7ed09893c982b27c0a61f8fd49689fea9d4" Jul 06 20:02:53.045749 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:53.045718 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"319fc5d3cb374f87358981990f49c7ed09893c982b27c0a61f8fd49689fea9d4"} err="failed to get container status \"319fc5d3cb374f87358981990f49c7ed09893c982b27c0a61f8fd49689fea9d4\": rpc error: code = NotFound desc = could not find container \"319fc5d3cb374f87358981990f49c7ed09893c982b27c0a61f8fd49689fea9d4\": container with ID starting with 319fc5d3cb374f87358981990f49c7ed09893c982b27c0a61f8fd49689fea9d4 not found: ID does not exist" Jul 06 20:02:53.050708 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:53.050679 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9"] Jul 06 20:02:53.055604 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:53.055582 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f1eed6-openshift-default-68ffc86465-24tc9"] Jul 06 20:02:53.690040 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:53.690002 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92422236-ab8d-44b4-8d7b-263b8490dca7" path="/var/lib/kubelet/pods/92422236-ab8d-44b4-8d7b-263b8490dca7/volumes" Jul 06 20:02:54.837510 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:54.837476 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf"] Jul 06 20:02:54.837949 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:54.837692 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" podUID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" containerName="istio-proxy" containerID="cri-o://5e508e5e980ba216cc739faf5743ad08642642736ceb93abd52fcd12f18f3e6d" gracePeriod=30 Jul 06 20:02:55.784047 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.784016 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj"] Jul 06 20:02:55.784428 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.784415 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92422236-ab8d-44b4-8d7b-263b8490dca7" containerName="istio-proxy" Jul 06 20:02:55.784488 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.784430 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="92422236-ab8d-44b4-8d7b-263b8490dca7" containerName="istio-proxy" Jul 06 20:02:55.784529 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.784509 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="92422236-ab8d-44b4-8d7b-263b8490dca7" containerName="istio-proxy" Jul 06 20:02:55.788662 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.788645 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:55.791194 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.791174 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-4430fb-openshift-default-dockercfg-nfb48\"" Jul 06 20:02:55.803470 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.803449 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj"] Jul 06 20:02:55.936597 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.936557 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-data\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:55.937028 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.936604 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-credential-socket\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:55.937028 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.936661 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-podinfo\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:55.937028 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.936742 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-workload-socket\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:55.937028 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.936788 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-envoy\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:55.937028 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.936844 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-token\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:55.937028 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.936876 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-workload-certs\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:55.937028 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.936906 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2l59\" (UniqueName: \"kubernetes.io/projected/cfcc8084-4bb1-43a8-94ae-2770a53d2608-kube-api-access-p2l59\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:55.937028 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:55.936956 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istiod-ca-cert\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.038135 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.038061 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-token\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.038135 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.038094 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-workload-certs\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.038135 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.038131 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2l59\" (UniqueName: \"kubernetes.io/projected/cfcc8084-4bb1-43a8-94ae-2770a53d2608-kube-api-access-p2l59\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.038378 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.038174 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istiod-ca-cert\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.038378 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.038261 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-data\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.038378 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.038314 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-credential-socket\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.038378 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.038346 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-podinfo\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.038378 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.038373 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-workload-socket\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.038638 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.038412 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-envoy\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.038638 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.038563 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-workload-certs\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.038638 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.038609 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-data\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.038748 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.038689 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-workload-socket\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.038790 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.038775 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-credential-socket\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.038972 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.038952 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istiod-ca-cert\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.040623 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.040593 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-envoy\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.040735 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.040717 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-podinfo\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.060230 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.060205 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2l59\" (UniqueName: \"kubernetes.io/projected/cfcc8084-4bb1-43a8-94ae-2770a53d2608-kube-api-access-p2l59\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.062008 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.061988 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-token\") pod \"e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.098578 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.098558 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:56.237001 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.236976 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj"] Jul 06 20:02:56.238093 ip-10-0-141-33 kubenswrapper[2561]: W0706 20:02:56.238066 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfcc8084_4bb1_43a8_94ae_2770a53d2608.slice/crio-8b9f3665984efd48d5a6b8a00aee2061209ecb0dd192ef0c86f5b073671681db WatchSource:0}: Error finding container 8b9f3665984efd48d5a6b8a00aee2061209ecb0dd192ef0c86f5b073671681db: Status 404 returned error can't find the container with id 8b9f3665984efd48d5a6b8a00aee2061209ecb0dd192ef0c86f5b073671681db Jul 06 20:02:56.240201 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.240172 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:56.240259 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.240247 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:56.240332 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:56.240316 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:02:57.051832 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:57.051799 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" event={"ID":"cfcc8084-4bb1-43a8-94ae-2770a53d2608","Type":"ContainerStarted","Data":"ba1150fcfd32f442033d228caf8d33494585168d63486827cf598a1bebb7ee7f"} Jul 06 20:02:57.052202 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:57.051853 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" event={"ID":"cfcc8084-4bb1-43a8-94ae-2770a53d2608","Type":"ContainerStarted","Data":"8b9f3665984efd48d5a6b8a00aee2061209ecb0dd192ef0c86f5b073671681db"} Jul 06 20:02:57.073674 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:57.073625 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" podStartSLOduration=2.073606908 podStartE2EDuration="2.073606908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:02:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:02:57.071763295 +0000 UTC m=+2111.939438855" watchObservedRunningTime="2026-07-06 20:02:57.073606908 +0000 UTC m=+2111.941282464" Jul 06 20:02:57.098965 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:57.098933 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:57.103575 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:57.103550 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:58.056345 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:58.056317 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:02:58.057370 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:02:58.057349 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:03:00.072135 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.072100 2561 generic.go:356] "Generic (PLEG): container finished" podID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" containerID="5e508e5e980ba216cc739faf5743ad08642642736ceb93abd52fcd12f18f3e6d" exitCode=0 Jul 06 20:03:00.072526 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.072191 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" event={"ID":"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6","Type":"ContainerDied","Data":"5e508e5e980ba216cc739faf5743ad08642642736ceb93abd52fcd12f18f3e6d"} Jul 06 20:03:00.072526 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.072248 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" event={"ID":"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6","Type":"ContainerDied","Data":"7f5fc21cf77ba9dba3b8c16e2674cba6c010db0c62e5ec14375e8034e1760828"} Jul 06 20:03:00.072526 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.072268 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f5fc21cf77ba9dba3b8c16e2674cba6c010db0c62e5ec14375e8034e1760828" Jul 06 20:03:00.072526 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.072283 2561 scope.go:117] "RemoveContainer" containerID="0e4de81e12952e83e23fc2052c5ceb96329a233c78ee8f8d90a1e57d0b810afa" Jul 06 20:03:00.075122 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.075101 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:03:00.178331 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.178309 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-podinfo\") pod \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " Jul 06 20:03:00.178497 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.178352 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nbfj\" (UniqueName: \"kubernetes.io/projected/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-kube-api-access-7nbfj\") pod \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " Jul 06 20:03:00.178497 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.178386 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-workload-socket\") pod \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " Jul 06 20:03:00.178497 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.178436 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-data\") pod \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " Jul 06 20:03:00.178648 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.178507 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-envoy\") pod \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " Jul 06 20:03:00.178648 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.178558 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-credential-socket\") pod \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " Jul 06 20:03:00.178648 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.178612 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-token\") pod \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " Jul 06 20:03:00.178648 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.178626 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" (UID: "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:03:00.178851 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.178635 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-workload-certs\") pod \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " Jul 06 20:03:00.178851 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.178732 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istiod-ca-cert\") pod \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\" (UID: \"44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6\") " Jul 06 20:03:00.178851 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.178826 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" (UID: "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:03:00.178851 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.178840 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" (UID: "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:03:00.179468 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.179187 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" (UID: "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:03:00.179468 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.179208 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-credential-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:03:00.179468 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.179228 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-workload-certs\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:03:00.179468 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.179240 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-workload-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:03:00.179468 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.179413 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-data" (OuterVolumeSpecName: "istio-data") pod "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" (UID: "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:03:00.181000 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.180972 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" (UID: "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:03:00.181121 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.181045 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-kube-api-access-7nbfj" (OuterVolumeSpecName: "kube-api-access-7nbfj") pod "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" (UID: "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6"). InnerVolumeSpecName "kube-api-access-7nbfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:03:00.181121 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.181056 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-token" (OuterVolumeSpecName: "istio-token") pod "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" (UID: "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:03:00.181121 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.181084 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" (UID: "44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:03:00.280473 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.280447 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istiod-ca-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:03:00.280473 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.280469 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-podinfo\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:03:00.280642 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.280481 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nbfj\" (UniqueName: \"kubernetes.io/projected/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-kube-api-access-7nbfj\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:03:00.280642 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.280489 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-data\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:03:00.280642 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.280497 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-envoy\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:03:00.280642 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:00.280506 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6-istio-token\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:03:01.079147 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:01.079121 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf" Jul 06 20:03:01.116895 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:01.116866 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf"] Jul 06 20:03:01.122517 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:01.122495 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-0e21dc-openshift-default-554c896987-z6tkf"] Jul 06 20:03:01.686829 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:01.686797 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" path="/var/lib/kubelet/pods/44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6/volumes" Jul 06 20:03:04.269472 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.269431 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq"] Jul 06 20:03:04.270047 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.270026 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" containerName="istio-proxy" Jul 06 20:03:04.270169 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.270049 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" containerName="istio-proxy" Jul 06 20:03:04.270169 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.270063 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92422236-ab8d-44b4-8d7b-263b8490dca7" containerName="istio-proxy" Jul 06 20:03:04.270169 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.270071 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="92422236-ab8d-44b4-8d7b-263b8490dca7" containerName="istio-proxy" Jul 06 20:03:04.270169 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.270084 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" containerName="istio-proxy" Jul 06 20:03:04.270169 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.270091 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" containerName="istio-proxy" Jul 06 20:03:04.270434 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.270208 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" containerName="istio-proxy" Jul 06 20:03:04.270434 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.270222 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="92422236-ab8d-44b4-8d7b-263b8490dca7" containerName="istio-proxy" Jul 06 20:03:04.274784 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.274764 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.277105 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.277085 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-33ae61-openshift-default-dockercfg-q8s42\"" Jul 06 20:03:04.290531 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.290502 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq"] Jul 06 20:03:04.420908 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.420874 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-data\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.421110 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.420926 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-workload-socket\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.421110 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.421036 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-podinfo\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.421110 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.421075 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tpfw\" (UniqueName: \"kubernetes.io/projected/63711c2a-546f-4a0b-972e-6da42ae718d8-kube-api-access-8tpfw\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.421242 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.421152 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-envoy\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.421242 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.421208 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-token\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.421358 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.421245 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-workload-certs\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.421358 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.421266 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63711c2a-546f-4a0b-972e-6da42ae718d8-istiod-ca-cert\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.421358 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.421284 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-credential-socket\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.522705 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.522595 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-workload-socket\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.522890 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.522815 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-podinfo\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.522890 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.522855 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tpfw\" (UniqueName: \"kubernetes.io/projected/63711c2a-546f-4a0b-972e-6da42ae718d8-kube-api-access-8tpfw\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.523008 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.522906 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-envoy\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.523008 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.522962 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-token\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.523112 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.523004 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-workload-certs\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.523112 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.523035 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63711c2a-546f-4a0b-972e-6da42ae718d8-istiod-ca-cert\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.523112 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.523066 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-credential-socket\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.523244 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.523117 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-data\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.523596 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.523567 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-workload-certs\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.523921 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.523893 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-credential-socket\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.524018 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.523930 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63711c2a-546f-4a0b-972e-6da42ae718d8-istiod-ca-cert\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.524082 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.524019 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-workload-socket\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.524261 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.524238 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-data\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.526654 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.526131 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-envoy\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.526654 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.526281 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-podinfo\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.532702 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.532664 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-token\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.533438 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.533396 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tpfw\" (UniqueName: \"kubernetes.io/projected/63711c2a-546f-4a0b-972e-6da42ae718d8-kube-api-access-8tpfw\") pod \"e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.587429 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.587398 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:04.754417 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.752020 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq"] Jul 06 20:03:04.759388 ip-10-0-141-33 kubenswrapper[2561]: W0706 20:03:04.759327 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63711c2a_546f_4a0b_972e_6da42ae718d8.slice/crio-548b1b2388b6181997c6aec9d6c53571c89c63b01ef6e8d07d306a3f4b43bf84 WatchSource:0}: Error finding container 548b1b2388b6181997c6aec9d6c53571c89c63b01ef6e8d07d306a3f4b43bf84: Status 404 returned error can't find the container with id 548b1b2388b6181997c6aec9d6c53571c89c63b01ef6e8d07d306a3f4b43bf84 Jul 06 20:03:04.761713 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.761684 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:03:04.761856 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.761842 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:03:04.761941 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:04.761930 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:03:05.102943 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:05.102853 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" event={"ID":"63711c2a-546f-4a0b-972e-6da42ae718d8","Type":"ContainerStarted","Data":"fc10f76fb7ff6bcb0a1ffe04cbf5cec18ab69bfbe43d25ae1473d15e44569317"} Jul 06 20:03:05.102943 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:05.102924 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" event={"ID":"63711c2a-546f-4a0b-972e-6da42ae718d8","Type":"ContainerStarted","Data":"548b1b2388b6181997c6aec9d6c53571c89c63b01ef6e8d07d306a3f4b43bf84"} Jul 06 20:03:05.127726 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:05.127675 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" podStartSLOduration=1.127659883 podStartE2EDuration="1.127659883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:03:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:03:05.124467156 +0000 UTC m=+2119.992142713" watchObservedRunningTime="2026-07-06 20:03:05.127659883 +0000 UTC m=+2119.995335437" Jul 06 20:03:05.588550 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:05.588516 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:05.593110 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:05.593084 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:06.107864 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:06.107832 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:06.108862 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:06.108846 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:11.138022 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:11.137995 2561 generic.go:356] "Generic (PLEG): container finished" podID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" containerID="ba1150fcfd32f442033d228caf8d33494585168d63486827cf598a1bebb7ee7f" exitCode=0 Jul 06 20:03:11.138435 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:11.138072 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" event={"ID":"cfcc8084-4bb1-43a8-94ae-2770a53d2608","Type":"ContainerDied","Data":"ba1150fcfd32f442033d228caf8d33494585168d63486827cf598a1bebb7ee7f"} Jul 06 20:03:11.138507 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:11.138493 2561 scope.go:117] "RemoveContainer" containerID="ba1150fcfd32f442033d228caf8d33494585168d63486827cf598a1bebb7ee7f" Jul 06 20:03:11.139525 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:11.139494 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:03:11.139616 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:11.139571 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:03:11.139616 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:11.139605 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:03:12.150004 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:12.149946 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" event={"ID":"cfcc8084-4bb1-43a8-94ae-2770a53d2608","Type":"ContainerStarted","Data":"d8dab081a678d2026994e5368f8202d89de2740dda25c3ecdcd14a1f1e33b0aa"} Jul 06 20:03:13.056627 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:13.056583 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:03:13.099715 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:13.099675 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:03:13.100946 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:13.100919 2561 patch_prober.go:28] interesting pod/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 06 20:03:13.101065 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:13.100964 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" podUID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 06 20:03:14.099817 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:14.099777 2561 patch_prober.go:28] interesting pod/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 06 20:03:14.100317 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:14.099851 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" podUID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 06 20:03:15.099022 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:15.098981 2561 patch_prober.go:28] interesting pod/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 06 20:03:15.099178 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:15.099045 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" podUID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 06 20:03:16.098924 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:16.098883 2561 patch_prober.go:28] interesting pod/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 06 20:03:16.099341 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:16.098937 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" podUID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 06 20:03:17.103050 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:17.103019 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:03:17.104259 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:17.104232 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:03:19.197963 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:19.197933 2561 generic.go:356] "Generic (PLEG): container finished" podID="63711c2a-546f-4a0b-972e-6da42ae718d8" containerID="fc10f76fb7ff6bcb0a1ffe04cbf5cec18ab69bfbe43d25ae1473d15e44569317" exitCode=0 Jul 06 20:03:19.197963 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:19.197971 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" event={"ID":"63711c2a-546f-4a0b-972e-6da42ae718d8","Type":"ContainerDied","Data":"fc10f76fb7ff6bcb0a1ffe04cbf5cec18ab69bfbe43d25ae1473d15e44569317"} Jul 06 20:03:19.198496 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:19.198432 2561 scope.go:117] "RemoveContainer" containerID="fc10f76fb7ff6bcb0a1ffe04cbf5cec18ab69bfbe43d25ae1473d15e44569317" Jul 06 20:03:19.199474 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:19.199442 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:03:19.199535 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:19.199514 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:03:19.199575 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:19.199553 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 20:03:20.204663 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:20.204624 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" event={"ID":"63711c2a-546f-4a0b-972e-6da42ae718d8","Type":"ContainerStarted","Data":"0520ba7e052ca7b82e6cb8827c152552daa6a4eb396e187d3eaf9e98abcb07f9"} Jul 06 20:03:20.588154 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:20.588060 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:20.589156 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:20.589130 2561 patch_prober.go:28] interesting pod/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 06 20:03:20.589246 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:20.589191 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" podUID="63711c2a-546f-4a0b-972e-6da42ae718d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 06 20:03:21.108539 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:21.108504 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:21.587996 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:21.587959 2561 patch_prober.go:28] interesting pod/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 06 20:03:21.588418 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:21.588026 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" podUID="63711c2a-546f-4a0b-972e-6da42ae718d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 06 20:03:22.587773 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:22.587730 2561 patch_prober.go:28] interesting pod/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 06 20:03:22.587984 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:22.587797 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" podUID="63711c2a-546f-4a0b-972e-6da42ae718d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 06 20:03:23.587760 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:23.587723 2561 patch_prober.go:28] interesting pod/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 06 20:03:23.588181 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:23.587794 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" podUID="63711c2a-546f-4a0b-972e-6da42ae718d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 06 20:03:24.592193 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:24.592154 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:03:24.594145 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:03:24.594119 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:08:45.848794 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:08:45.848764 2561 scope.go:117] "RemoveContainer" containerID="5e508e5e980ba216cc739faf5743ad08642642736ceb93abd52fcd12f18f3e6d" Jul 06 20:09:15.761092 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:15.761060 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj"] Jul 06 20:09:15.761649 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:15.761250 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" podUID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" containerName="istio-proxy" containerID="cri-o://d8dab081a678d2026994e5368f8202d89de2740dda25c3ecdcd14a1f1e33b0aa" gracePeriod=30 Jul 06 20:09:17.105435 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:17.105398 2561 patch_prober.go:28] interesting pod/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 20:09:17.105822 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:17.105461 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" podUID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 20:09:21.023709 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.023637 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:09:21.058116 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.058078 2561 generic.go:356] "Generic (PLEG): container finished" podID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" containerID="d8dab081a678d2026994e5368f8202d89de2740dda25c3ecdcd14a1f1e33b0aa" exitCode=0 Jul 06 20:09:21.058268 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.058148 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" Jul 06 20:09:21.058268 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.058163 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" event={"ID":"cfcc8084-4bb1-43a8-94ae-2770a53d2608","Type":"ContainerDied","Data":"d8dab081a678d2026994e5368f8202d89de2740dda25c3ecdcd14a1f1e33b0aa"} Jul 06 20:09:21.058268 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.058222 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj" event={"ID":"cfcc8084-4bb1-43a8-94ae-2770a53d2608","Type":"ContainerDied","Data":"8b9f3665984efd48d5a6b8a00aee2061209ecb0dd192ef0c86f5b073671681db"} Jul 06 20:09:21.058268 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.058246 2561 scope.go:117] "RemoveContainer" containerID="d8dab081a678d2026994e5368f8202d89de2740dda25c3ecdcd14a1f1e33b0aa" Jul 06 20:09:21.067803 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.067782 2561 scope.go:117] "RemoveContainer" containerID="ba1150fcfd32f442033d228caf8d33494585168d63486827cf598a1bebb7ee7f" Jul 06 20:09:21.077067 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.077046 2561 scope.go:117] "RemoveContainer" containerID="d8dab081a678d2026994e5368f8202d89de2740dda25c3ecdcd14a1f1e33b0aa" Jul 06 20:09:21.077397 ip-10-0-141-33 kubenswrapper[2561]: E0706 20:09:21.077369 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8dab081a678d2026994e5368f8202d89de2740dda25c3ecdcd14a1f1e33b0aa\": container with ID starting with d8dab081a678d2026994e5368f8202d89de2740dda25c3ecdcd14a1f1e33b0aa not found: ID does not exist" containerID="d8dab081a678d2026994e5368f8202d89de2740dda25c3ecdcd14a1f1e33b0aa" Jul 06 20:09:21.077466 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.077411 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8dab081a678d2026994e5368f8202d89de2740dda25c3ecdcd14a1f1e33b0aa"} err="failed to get container status \"d8dab081a678d2026994e5368f8202d89de2740dda25c3ecdcd14a1f1e33b0aa\": rpc error: code = NotFound desc = could not find container \"d8dab081a678d2026994e5368f8202d89de2740dda25c3ecdcd14a1f1e33b0aa\": container with ID starting with d8dab081a678d2026994e5368f8202d89de2740dda25c3ecdcd14a1f1e33b0aa not found: ID does not exist" Jul 06 20:09:21.077466 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.077430 2561 scope.go:117] "RemoveContainer" containerID="ba1150fcfd32f442033d228caf8d33494585168d63486827cf598a1bebb7ee7f" Jul 06 20:09:21.077734 ip-10-0-141-33 kubenswrapper[2561]: E0706 20:09:21.077707 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba1150fcfd32f442033d228caf8d33494585168d63486827cf598a1bebb7ee7f\": container with ID starting with ba1150fcfd32f442033d228caf8d33494585168d63486827cf598a1bebb7ee7f not found: ID does not exist" containerID="ba1150fcfd32f442033d228caf8d33494585168d63486827cf598a1bebb7ee7f" Jul 06 20:09:21.077804 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.077744 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba1150fcfd32f442033d228caf8d33494585168d63486827cf598a1bebb7ee7f"} err="failed to get container status \"ba1150fcfd32f442033d228caf8d33494585168d63486827cf598a1bebb7ee7f\": rpc error: code = NotFound desc = could not find container \"ba1150fcfd32f442033d228caf8d33494585168d63486827cf598a1bebb7ee7f\": container with ID starting with ba1150fcfd32f442033d228caf8d33494585168d63486827cf598a1bebb7ee7f not found: ID does not exist" Jul 06 20:09:21.193305 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.193259 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2l59\" (UniqueName: \"kubernetes.io/projected/cfcc8084-4bb1-43a8-94ae-2770a53d2608-kube-api-access-p2l59\") pod \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " Jul 06 20:09:21.193466 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.193340 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-credential-socket\") pod \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " Jul 06 20:09:21.193466 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.193383 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-token\") pod \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " Jul 06 20:09:21.193546 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.193465 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-workload-socket\") pod \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " Jul 06 20:09:21.193546 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.193503 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-envoy\") pod \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " Jul 06 20:09:21.193657 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.193555 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-podinfo\") pod \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " Jul 06 20:09:21.193657 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.193585 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-data\") pod \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " Jul 06 20:09:21.193657 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.193598 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cfcc8084-4bb1-43a8-94ae-2770a53d2608" (UID: "cfcc8084-4bb1-43a8-94ae-2770a53d2608"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:21.193807 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.193705 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-workload-certs\") pod \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " Jul 06 20:09:21.193807 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.193750 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istiod-ca-cert\") pod \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\" (UID: \"cfcc8084-4bb1-43a8-94ae-2770a53d2608\") " Jul 06 20:09:21.193915 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.193792 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cfcc8084-4bb1-43a8-94ae-2770a53d2608" (UID: "cfcc8084-4bb1-43a8-94ae-2770a53d2608"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:21.194040 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.193997 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cfcc8084-4bb1-43a8-94ae-2770a53d2608" (UID: "cfcc8084-4bb1-43a8-94ae-2770a53d2608"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:21.194348 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.194240 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-workload-certs\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:21.194348 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.194264 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-credential-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:21.194348 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.194280 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-workload-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:21.194348 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.194310 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cfcc8084-4bb1-43a8-94ae-2770a53d2608" (UID: "cfcc8084-4bb1-43a8-94ae-2770a53d2608"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:09:21.194626 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.194457 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-data" (OuterVolumeSpecName: "istio-data") pod "cfcc8084-4bb1-43a8-94ae-2770a53d2608" (UID: "cfcc8084-4bb1-43a8-94ae-2770a53d2608"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:21.195649 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.195624 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfcc8084-4bb1-43a8-94ae-2770a53d2608-kube-api-access-p2l59" (OuterVolumeSpecName: "kube-api-access-p2l59") pod "cfcc8084-4bb1-43a8-94ae-2770a53d2608" (UID: "cfcc8084-4bb1-43a8-94ae-2770a53d2608"). InnerVolumeSpecName "kube-api-access-p2l59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:09:21.195649 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.195644 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-token" (OuterVolumeSpecName: "istio-token") pod "cfcc8084-4bb1-43a8-94ae-2770a53d2608" (UID: "cfcc8084-4bb1-43a8-94ae-2770a53d2608"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:09:21.195814 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.195707 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cfcc8084-4bb1-43a8-94ae-2770a53d2608" (UID: "cfcc8084-4bb1-43a8-94ae-2770a53d2608"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:09:21.195814 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.195719 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cfcc8084-4bb1-43a8-94ae-2770a53d2608" (UID: "cfcc8084-4bb1-43a8-94ae-2770a53d2608"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:21.295774 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.295682 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-envoy\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:21.295774 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.295721 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-podinfo\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:21.295774 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.295742 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-data\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:21.295774 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.295751 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istiod-ca-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:21.295774 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.295760 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2l59\" (UniqueName: \"kubernetes.io/projected/cfcc8084-4bb1-43a8-94ae-2770a53d2608-kube-api-access-p2l59\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:21.295774 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.295770 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfcc8084-4bb1-43a8-94ae-2770a53d2608-istio-token\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:21.381822 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.381790 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj"] Jul 06 20:09:21.386394 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.386373 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-4430fb-openshift-default-69df56d9dc-mmgcj"] Jul 06 20:09:21.687759 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:21.687729 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" path="/var/lib/kubelet/pods/cfcc8084-4bb1-43a8-94ae-2770a53d2608/volumes" Jul 06 20:09:22.958052 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:22.958018 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq"] Jul 06 20:09:22.958490 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:22.958261 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" podUID="63711c2a-546f-4a0b-972e-6da42ae718d8" containerName="istio-proxy" containerID="cri-o://0520ba7e052ca7b82e6cb8827c152552daa6a4eb396e187d3eaf9e98abcb07f9" gracePeriod=30 Jul 06 20:09:24.595022 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:24.594980 2561 patch_prober.go:28] interesting pod/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 20:09:24.595419 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:24.595051 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" podUID="63711c2a-546f-4a0b-972e-6da42ae718d8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 20:09:28.094970 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.094941 2561 generic.go:356] "Generic (PLEG): container finished" podID="63711c2a-546f-4a0b-972e-6da42ae718d8" containerID="0520ba7e052ca7b82e6cb8827c152552daa6a4eb396e187d3eaf9e98abcb07f9" exitCode=0 Jul 06 20:09:28.095369 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.094998 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" event={"ID":"63711c2a-546f-4a0b-972e-6da42ae718d8","Type":"ContainerDied","Data":"0520ba7e052ca7b82e6cb8827c152552daa6a4eb396e187d3eaf9e98abcb07f9"} Jul 06 20:09:28.095369 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.095051 2561 scope.go:117] "RemoveContainer" containerID="fc10f76fb7ff6bcb0a1ffe04cbf5cec18ab69bfbe43d25ae1473d15e44569317" Jul 06 20:09:28.213152 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.213126 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:09:28.266588 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.266511 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-workload-certs\") pod \"63711c2a-546f-4a0b-972e-6da42ae718d8\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " Jul 06 20:09:28.266588 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.266562 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-podinfo\") pod \"63711c2a-546f-4a0b-972e-6da42ae718d8\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " Jul 06 20:09:28.266588 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.266585 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63711c2a-546f-4a0b-972e-6da42ae718d8-istiod-ca-cert\") pod \"63711c2a-546f-4a0b-972e-6da42ae718d8\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " Jul 06 20:09:28.266852 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.266648 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tpfw\" (UniqueName: \"kubernetes.io/projected/63711c2a-546f-4a0b-972e-6da42ae718d8-kube-api-access-8tpfw\") pod \"63711c2a-546f-4a0b-972e-6da42ae718d8\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " Jul 06 20:09:28.266852 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.266716 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-data\") pod \"63711c2a-546f-4a0b-972e-6da42ae718d8\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " Jul 06 20:09:28.266852 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.266760 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-envoy\") pod \"63711c2a-546f-4a0b-972e-6da42ae718d8\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " Jul 06 20:09:28.266852 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.266782 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-credential-socket\") pod \"63711c2a-546f-4a0b-972e-6da42ae718d8\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " Jul 06 20:09:28.266852 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.266793 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "63711c2a-546f-4a0b-972e-6da42ae718d8" (UID: "63711c2a-546f-4a0b-972e-6da42ae718d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:28.266852 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.266806 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-token\") pod \"63711c2a-546f-4a0b-972e-6da42ae718d8\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " Jul 06 20:09:28.267151 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.266866 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-workload-socket\") pod \"63711c2a-546f-4a0b-972e-6da42ae718d8\" (UID: \"63711c2a-546f-4a0b-972e-6da42ae718d8\") " Jul 06 20:09:28.267203 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.267154 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63711c2a-546f-4a0b-972e-6da42ae718d8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "63711c2a-546f-4a0b-972e-6da42ae718d8" (UID: "63711c2a-546f-4a0b-972e-6da42ae718d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:09:28.267308 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.267267 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-workload-certs\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:28.267405 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.267373 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "63711c2a-546f-4a0b-972e-6da42ae718d8" (UID: "63711c2a-546f-4a0b-972e-6da42ae718d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:28.267540 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.267494 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "63711c2a-546f-4a0b-972e-6da42ae718d8" (UID: "63711c2a-546f-4a0b-972e-6da42ae718d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:28.267744 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.267720 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-data" (OuterVolumeSpecName: "istio-data") pod "63711c2a-546f-4a0b-972e-6da42ae718d8" (UID: "63711c2a-546f-4a0b-972e-6da42ae718d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:28.269070 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.269045 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "63711c2a-546f-4a0b-972e-6da42ae718d8" (UID: "63711c2a-546f-4a0b-972e-6da42ae718d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:28.269265 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.269247 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "63711c2a-546f-4a0b-972e-6da42ae718d8" (UID: "63711c2a-546f-4a0b-972e-6da42ae718d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:09:28.269402 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.269380 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-token" (OuterVolumeSpecName: "istio-token") pod "63711c2a-546f-4a0b-972e-6da42ae718d8" (UID: "63711c2a-546f-4a0b-972e-6da42ae718d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:09:28.269451 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.269380 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63711c2a-546f-4a0b-972e-6da42ae718d8-kube-api-access-8tpfw" (OuterVolumeSpecName: "kube-api-access-8tpfw") pod "63711c2a-546f-4a0b-972e-6da42ae718d8" (UID: "63711c2a-546f-4a0b-972e-6da42ae718d8"). InnerVolumeSpecName "kube-api-access-8tpfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:09:28.368772 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.368723 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-data\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:28.368772 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.368763 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-envoy\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:28.368772 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.368774 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-credential-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:28.368772 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.368784 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-token\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:28.368772 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.368794 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63711c2a-546f-4a0b-972e-6da42ae718d8-workload-socket\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:28.369081 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.368801 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63711c2a-546f-4a0b-972e-6da42ae718d8-istio-podinfo\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:28.369081 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.368809 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63711c2a-546f-4a0b-972e-6da42ae718d8-istiod-ca-cert\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:28.369081 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:28.368817 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tpfw\" (UniqueName: \"kubernetes.io/projected/63711c2a-546f-4a0b-972e-6da42ae718d8-kube-api-access-8tpfw\") on node \"ip-10-0-141-33.ec2.internal\" DevicePath \"\"" Jul 06 20:09:29.100930 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:29.100894 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" event={"ID":"63711c2a-546f-4a0b-972e-6da42ae718d8","Type":"ContainerDied","Data":"548b1b2388b6181997c6aec9d6c53571c89c63b01ef6e8d07d306a3f4b43bf84"} Jul 06 20:09:29.100930 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:29.100910 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq" Jul 06 20:09:29.101483 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:29.100952 2561 scope.go:117] "RemoveContainer" containerID="0520ba7e052ca7b82e6cb8827c152552daa6a4eb396e187d3eaf9e98abcb07f9" Jul 06 20:09:29.123459 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:29.123435 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq"] Jul 06 20:09:29.127321 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:29.127284 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-33ae61-openshift-default-8f54c8fff-lk2mq"] Jul 06 20:09:29.687153 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:29.687113 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63711c2a-546f-4a0b-972e-6da42ae718d8" path="/var/lib/kubelet/pods/63711c2a-546f-4a0b-972e-6da42ae718d8/volumes" Jul 06 20:09:29.846405 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:29.846376 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57b585888c-cwgsz_945a5232-1920-431c-a151-e36544a90d6f/copy-manifests/0.log" Jul 06 20:09:29.853590 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:29.853572 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57b585888c-cwgsz_945a5232-1920-431c-a151-e36544a90d6f/manager/0.log" Jul 06 20:09:32.678554 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:32.678520 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf479jh6_8c8354b6-b95c-4ac1-bf88-f9eb8029678b/istio-proxy/0.log" Jul 06 20:09:32.805820 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:32.805793 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p8prq_a005f86f-7fab-4720-af71-c49804fee909/discovery/0.log" Jul 06 20:09:33.179665 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:33.179637 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cs4bk_c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e/istio-proxy/1.log" Jul 06 20:09:33.317719 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:33.317688 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-tnkpv_a4a687a8-687d-490d-887d-6635e416ae2e/payload-pre-processing/0.log" Jul 06 20:09:34.053012 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:34.052983 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-68kz6_bdf077ab-76cd-42de-8e90-a47052658963/main/0.log" Jul 06 20:09:34.170964 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:34.170933 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-69hl8_b7ed9752-50da-40ea-ab55-30ec35f86f5f/main/0.log" Jul 06 20:09:34.415167 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:34.415085 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-lz5vb_d7e266be-03d6-45b5-977b-197a93d3d042/main/0.log" Jul 06 20:09:41.757960 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:41.757885 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ddpss_4d150905-fd23-4a84-9a45-729d76fd6652/global-pull-secret-syncer/0.log" Jul 06 20:09:41.900199 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:41.900172 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-klbgj_90559109-2258-4954-8b4c-c99a0e8aff98/konnectivity-agent/0.log" Jul 06 20:09:42.031375 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:42.031284 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-33.ec2.internal_85b0a6ebff246af6adbee180d5545e92/haproxy/0.log" Jul 06 20:09:49.283015 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.282930 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_39c14ca0-db08-4cbe-b760-720e9c6210a1/alertmanager/0.log" Jul 06 20:09:49.314201 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.314176 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_39c14ca0-db08-4cbe-b760-720e9c6210a1/config-reloader/0.log" Jul 06 20:09:49.344955 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.344936 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_39c14ca0-db08-4cbe-b760-720e9c6210a1/kube-rbac-proxy-web/0.log" Jul 06 20:09:49.376716 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.376694 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_39c14ca0-db08-4cbe-b760-720e9c6210a1/kube-rbac-proxy/0.log" Jul 06 20:09:49.412326 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.412284 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_39c14ca0-db08-4cbe-b760-720e9c6210a1/kube-rbac-proxy-metric/0.log" Jul 06 20:09:49.444373 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.444348 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_39c14ca0-db08-4cbe-b760-720e9c6210a1/prom-label-proxy/0.log" Jul 06 20:09:49.476496 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.476475 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_39c14ca0-db08-4cbe-b760-720e9c6210a1/init-config-reloader/0.log" Jul 06 20:09:49.563456 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.563376 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-q8v7b_66d06755-9fb1-4361-ae12-04286b7be77c/kube-state-metrics/0.log" Jul 06 20:09:49.588846 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.588819 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-q8v7b_66d06755-9fb1-4361-ae12-04286b7be77c/kube-rbac-proxy-main/0.log" Jul 06 20:09:49.612581 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.612551 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-q8v7b_66d06755-9fb1-4361-ae12-04286b7be77c/kube-rbac-proxy-self/0.log" Jul 06 20:09:49.644502 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.644479 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5c5446d845-trq9w_e2009547-e35a-4b33-aed3-a89b82d24541/metrics-server/0.log" Jul 06 20:09:49.679506 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.679482 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-4c629_d4b7c468-aa3f-4188-a9bc-5d09b43dcf2d/monitoring-plugin/0.log" Jul 06 20:09:49.727517 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.727491 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-62b95_6b97f973-3964-4944-af29-163662153573/node-exporter/0.log" Jul 06 20:09:49.754259 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.754238 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-62b95_6b97f973-3964-4944-af29-163662153573/kube-rbac-proxy/0.log" Jul 06 20:09:49.778554 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.778531 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-62b95_6b97f973-3964-4944-af29-163662153573/init-textfile/0.log" Jul 06 20:09:49.977922 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:49.977890 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-8gkq5_3c302db2-20ca-4d3c-91ff-5ce9c6565adf/kube-rbac-proxy-main/0.log" Jul 06 20:09:50.005756 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.005730 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-8gkq5_3c302db2-20ca-4d3c-91ff-5ce9c6565adf/kube-rbac-proxy-self/0.log" Jul 06 20:09:50.028122 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.028096 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-8gkq5_3c302db2-20ca-4d3c-91ff-5ce9c6565adf/openshift-state-metrics/0.log" Jul 06 20:09:50.067272 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.067248 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f88086f4-aa96-41dc-9ac4-a32d5af1c4a8/prometheus/0.log" Jul 06 20:09:50.088985 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.088962 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f88086f4-aa96-41dc-9ac4-a32d5af1c4a8/config-reloader/0.log" Jul 06 20:09:50.113105 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.113076 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f88086f4-aa96-41dc-9ac4-a32d5af1c4a8/thanos-sidecar/0.log" Jul 06 20:09:50.137282 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.137259 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f88086f4-aa96-41dc-9ac4-a32d5af1c4a8/kube-rbac-proxy-web/0.log" Jul 06 20:09:50.164274 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.164252 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f88086f4-aa96-41dc-9ac4-a32d5af1c4a8/kube-rbac-proxy/0.log" Jul 06 20:09:50.188984 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.188962 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f88086f4-aa96-41dc-9ac4-a32d5af1c4a8/kube-rbac-proxy-thanos/0.log" Jul 06 20:09:50.212495 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.212466 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f88086f4-aa96-41dc-9ac4-a32d5af1c4a8/init-config-reloader/0.log" Jul 06 20:09:50.244024 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.243948 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-29kfp_ec9d6541-fd28-44b5-9746-4f8ca3cde847/prometheus-operator/0.log" Jul 06 20:09:50.266657 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.266607 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-29kfp_ec9d6541-fd28-44b5-9746-4f8ca3cde847/kube-rbac-proxy/0.log" Jul 06 20:09:50.328944 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.328920 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-689b96b8dd-sz8r7_208218e1-8f44-408c-9e2f-a4484b7045f5/telemeter-client/0.log" Jul 06 20:09:50.352527 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.352486 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-689b96b8dd-sz8r7_208218e1-8f44-408c-9e2f-a4484b7045f5/reload/0.log" Jul 06 20:09:50.375634 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.375613 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-689b96b8dd-sz8r7_208218e1-8f44-408c-9e2f-a4484b7045f5/kube-rbac-proxy/0.log" Jul 06 20:09:50.412864 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.412844 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5856747687-dl7fq_4f267dd3-a02f-46f5-8041-b2c06becc979/thanos-query/0.log" Jul 06 20:09:50.436664 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.436646 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5856747687-dl7fq_4f267dd3-a02f-46f5-8041-b2c06becc979/kube-rbac-proxy-web/0.log" Jul 06 20:09:50.458397 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.458378 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5856747687-dl7fq_4f267dd3-a02f-46f5-8041-b2c06becc979/kube-rbac-proxy/0.log" Jul 06 20:09:50.481059 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.481039 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5856747687-dl7fq_4f267dd3-a02f-46f5-8041-b2c06becc979/prom-label-proxy/0.log" Jul 06 20:09:50.503148 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.503074 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5856747687-dl7fq_4f267dd3-a02f-46f5-8041-b2c06becc979/kube-rbac-proxy-rules/0.log" Jul 06 20:09:50.528514 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:50.528488 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5856747687-dl7fq_4f267dd3-a02f-46f5-8041-b2c06becc979/kube-rbac-proxy-metrics/0.log" Jul 06 20:09:51.778600 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.778565 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l"] Jul 06 20:09:51.779081 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.779061 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" containerName="istio-proxy" Jul 06 20:09:51.779154 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.779084 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" containerName="istio-proxy" Jul 06 20:09:51.779154 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.779097 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" containerName="istio-proxy" Jul 06 20:09:51.779154 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.779106 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" containerName="istio-proxy" Jul 06 20:09:51.779154 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.779116 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63711c2a-546f-4a0b-972e-6da42ae718d8" containerName="istio-proxy" Jul 06 20:09:51.779154 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.779124 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="63711c2a-546f-4a0b-972e-6da42ae718d8" containerName="istio-proxy" Jul 06 20:09:51.779154 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.779152 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63711c2a-546f-4a0b-972e-6da42ae718d8" containerName="istio-proxy" Jul 06 20:09:51.779154 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.779158 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="63711c2a-546f-4a0b-972e-6da42ae718d8" containerName="istio-proxy" Jul 06 20:09:51.779497 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.779233 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" containerName="istio-proxy" Jul 06 20:09:51.779497 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.779247 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfcc8084-4bb1-43a8-94ae-2770a53d2608" containerName="istio-proxy" Jul 06 20:09:51.779497 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.779258 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e8ab0b-1a75-4d0e-9c9c-3ba9dfd3feb6" containerName="istio-proxy" Jul 06 20:09:51.779497 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.779272 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="63711c2a-546f-4a0b-972e-6da42ae718d8" containerName="istio-proxy" Jul 06 20:09:51.779497 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.779280 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="63711c2a-546f-4a0b-972e-6da42ae718d8" containerName="istio-proxy" Jul 06 20:09:51.784275 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.784255 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:51.786553 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.786529 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f6s5f\"/\"kube-root-ca.crt\"" Jul 06 20:09:51.786684 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.786562 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f6s5f\"/\"openshift-service-ca.crt\"" Jul 06 20:09:51.787083 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.787061 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f6s5f\"/\"default-dockercfg-p6z85\"" Jul 06 20:09:51.794173 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.794146 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l"] Jul 06 20:09:51.898264 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.898231 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/abf7b37b-bfc0-4650-af13-ec9fb866d669-lib-modules\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:51.898446 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.898274 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/abf7b37b-bfc0-4650-af13-ec9fb866d669-podres\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:51.898446 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.898377 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk7p2\" (UniqueName: \"kubernetes.io/projected/abf7b37b-bfc0-4650-af13-ec9fb866d669-kube-api-access-wk7p2\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:51.898446 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.898437 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abf7b37b-bfc0-4650-af13-ec9fb866d669-sys\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:51.898574 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.898491 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/abf7b37b-bfc0-4650-af13-ec9fb866d669-proc\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:51.999079 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.999040 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/abf7b37b-bfc0-4650-af13-ec9fb866d669-podres\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:51.999239 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.999105 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk7p2\" (UniqueName: \"kubernetes.io/projected/abf7b37b-bfc0-4650-af13-ec9fb866d669-kube-api-access-wk7p2\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:51.999239 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.999153 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abf7b37b-bfc0-4650-af13-ec9fb866d669-sys\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:51.999239 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.999209 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/abf7b37b-bfc0-4650-af13-ec9fb866d669-proc\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:51.999384 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.999248 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/abf7b37b-bfc0-4650-af13-ec9fb866d669-lib-modules\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:51.999384 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.999255 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/abf7b37b-bfc0-4650-af13-ec9fb866d669-proc\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:51.999384 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.999260 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abf7b37b-bfc0-4650-af13-ec9fb866d669-sys\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:51.999384 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.999206 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/abf7b37b-bfc0-4650-af13-ec9fb866d669-podres\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:51.999526 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:51.999411 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/abf7b37b-bfc0-4650-af13-ec9fb866d669-lib-modules\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:52.008439 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:52.008412 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk7p2\" (UniqueName: \"kubernetes.io/projected/abf7b37b-bfc0-4650-af13-ec9fb866d669-kube-api-access-wk7p2\") pod \"perf-node-gather-daemonset-qcl5l\" (UID: \"abf7b37b-bfc0-4650-af13-ec9fb866d669\") " pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:52.095885 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:52.095804 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:52.227255 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:52.227229 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l"] Jul 06 20:09:52.228844 ip-10-0-141-33 kubenswrapper[2561]: W0706 20:09:52.228808 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podabf7b37b_bfc0_4650_af13_ec9fb866d669.slice/crio-771f85ea5cee00e2f9e9c652c11b7c3c260364d03c3db831976b7c1c8c168769 WatchSource:0}: Error finding container 771f85ea5cee00e2f9e9c652c11b7c3c260364d03c3db831976b7c1c8c168769: Status 404 returned error can't find the container with id 771f85ea5cee00e2f9e9c652c11b7c3c260364d03c3db831976b7c1c8c168769 Jul 06 20:09:52.927940 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:52.927916 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6d84765bc6-zpmzq_e662b5d2-0861-450e-a28c-7abb338c1ce7/console/0.log" Jul 06 20:09:53.221684 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:53.221579 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" event={"ID":"abf7b37b-bfc0-4650-af13-ec9fb866d669","Type":"ContainerStarted","Data":"5834005c5f6b7ccd5a9e25b758725206c2fb9f444a6a5fe20ccc429beb0424c0"} Jul 06 20:09:53.221684 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:53.221638 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:09:53.221684 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:53.221654 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" event={"ID":"abf7b37b-bfc0-4650-af13-ec9fb866d669","Type":"ContainerStarted","Data":"771f85ea5cee00e2f9e9c652c11b7c3c260364d03c3db831976b7c1c8c168769"} Jul 06 20:09:53.240054 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:53.240005 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" podStartSLOduration=2.239989658 podStartE2EDuration="2.239989658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:09:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:53.237482848 +0000 UTC m=+2528.105158405" watchObservedRunningTime="2026-07-06 20:09:53.239989658 +0000 UTC m=+2528.107665213" Jul 06 20:09:54.783562 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:54.783531 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q9dm7_8233bc1c-af16-451c-bd31-29c9b3695124/dns/0.log" Jul 06 20:09:54.805492 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:54.805460 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q9dm7_8233bc1c-af16-451c-bd31-29c9b3695124/kube-rbac-proxy/0.log" Jul 06 20:09:54.924391 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:54.924362 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lg8j5_bc341ace-408b-47f4-8fbf-adaf13c4772f/dns-node-resolver/0.log" Jul 06 20:09:55.619178 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:55.619150 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rk8nh_766b167f-bf67-495e-86e7-1a06223c14ae/node-ca/0.log" Jul 06 20:09:56.564175 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:56.564138 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf479jh6_8c8354b6-b95c-4ac1-bf88-f9eb8029678b/istio-proxy/0.log" Jul 06 20:09:56.672059 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:56.672029 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p8prq_a005f86f-7fab-4720-af71-c49804fee909/discovery/0.log" Jul 06 20:09:56.750541 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:56.750512 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cs4bk_c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e/istio-proxy/0.log" Jul 06 20:09:56.777303 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:56.777272 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cs4bk_c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e/istio-proxy/1.log" Jul 06 20:09:56.809550 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:56.809525 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-tnkpv_a4a687a8-687d-490d-887d-6635e416ae2e/payload-pre-processing/0.log" Jul 06 20:09:57.647704 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:57.647675 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-84nwx_a9a4277f-542c-4748-a202-56ad0cf65ebd/serve-healthcheck-canary/0.log" Jul 06 20:09:58.335388 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:58.335357 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x66xp_825798dc-9fa7-44a5-8cd4-559fe23a6b94/kube-rbac-proxy/0.log" Jul 06 20:09:58.357198 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:58.357171 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x66xp_825798dc-9fa7-44a5-8cd4-559fe23a6b94/exporter/0.log" Jul 06 20:09:58.379625 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:58.379600 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x66xp_825798dc-9fa7-44a5-8cd4-559fe23a6b94/extractor/0.log" Jul 06 20:09:59.237130 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:09:59.237106 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-f6s5f/perf-node-gather-daemonset-qcl5l" Jul 06 20:10:01.534453 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:10:01.534371 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57b585888c-cwgsz_945a5232-1920-431c-a151-e36544a90d6f/manager/0.log" Jul 06 20:10:01.561961 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:10:01.561938 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57b585888c-cwgsz_945a5232-1920-431c-a151-e36544a90d6f/copy-manifests/0.log" Jul 06 20:10:02.486941 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:10:02.486907 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf479jh6_8c8354b6-b95c-4ac1-bf88-f9eb8029678b/istio-proxy/0.log" Jul 06 20:10:02.592661 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:10:02.592630 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p8prq_a005f86f-7fab-4720-af71-c49804fee909/discovery/0.log" Jul 06 20:10:02.640647 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:10:02.640609 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cs4bk_c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e/istio-proxy/0.log" Jul 06 20:10:02.672939 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:10:02.672908 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cs4bk_c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e/istio-proxy/1.log" Jul 06 20:10:02.694623 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:10:02.694597 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-tnkpv_a4a687a8-687d-490d-887d-6635e416ae2e/payload-pre-processing/0.log" Jul 06 20:10:03.008065 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:10:03.008034 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b7d6b4d5f-gldgz_2433e1cb-2cc2-4ba5-b04e-0f78b85d55e5/manager/0.log" Jul 06 20:10:03.062662 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:10:03.062633 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-krnrw_3447ee19-b873-4c82-8293-38d3be493a0f/openshift-lws-operator/0.log" Jul 06 20:10:04.026958 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:10:04.026927 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf479jh6_8c8354b6-b95c-4ac1-bf88-f9eb8029678b/istio-proxy/0.log" Jul 06 20:10:04.130464 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:10:04.130432 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p8prq_a005f86f-7fab-4720-af71-c49804fee909/discovery/0.log" Jul 06 20:10:04.181185 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:10:04.181155 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cs4bk_c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e/istio-proxy/0.log" Jul 06 20:10:04.207247 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:10:04.207213 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cs4bk_c9c03d15-1eb0-48d0-9d4f-98c8577f6d6e/istio-proxy/1.log" Jul 06 20:10:04.230119 ip-10-0-141-33 kubenswrapper[2561]: I0706 20:10:04.230081 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-tnkpv_a4a687a8-687d-490d-887d-6635e416ae2e/payload-pre-processing/0.log"